DATA MINING ISSUES

INTRODUCTION

Data mining is not an easy task, as the algorithms used can get very complex and data
is not always available at one place. It needs to be integrated from various heterogeneous data
sources. These factors also create some issues. Here in this tutorial, we will discuss the major
issues regarding

Mining Methodology and User Interaction

Performance Issues
Diverse Data Types Issues

The following diagram describes the major issues.

MINING METHODOLOGY AND USER INTERACTION ISSUES

It refers to the following kinds of issues

1
Mining different kinds of knowledge in databases

Different users may be interested in different kinds of knowledge. Therefore it is

necessary for data mining to cover a broad range of knowledge discovery task.

Interactive mining of knowledge at multiple levels of abstraction

The data mining process needs to be interactive because it allows users to focus the
search for patterns, providing and refining data mining requests based on the returned results.

Incorporation of background knowledge

To guide discovery process and to express the discovered patterns, the background
knowledge can be used. Background knowledge may be used to express the discovered
patterns not only in concise terms but at multiple levels of abstraction.

Data mining query languages and ad hoc data mining

Data Mining Query language that allows the user to describe ad hoc mining tasks,
should be integrated with a data warehouse query language and optimized for efficient and
flexible data mining.

Presentation and visualization of data mining results

Once the patterns are discovered it needs to be expressed in high level languages, and
visual representations. These representations should be easily understandable.

Handling noisy or incomplete data

The data cleaning methods are required to handle the noise and incomplete objects
while mining the data regularities. If the data cleaning methods are not there then the
accuracy of the discovered patterns will be poor.

Pattern evaluation

The patterns discovered should be interesting because either they represent common
knowledge or lack novelty.

2
 These reflect the kinds of knowledge mined, the ability to mine knowledge at multiple
granularities, the use of domain knowledge, ad hoc mining, and knowledge visualization.

Mining different kinds of knowledge databases: Data mining should cover a wide
spectrum of data analysis and knowledge discovery tasks, including data
characterization, discrimination, association, classification, clustering, tread and
deviation analysis, and similarity analysis.
Interactive mining of knowledge at multiple levels of abstraction: The data mining
process should be interactive. Interactive mining allows users to focus the search
for patterns, providing and refining data mining requests based on returned results.
Incorporation of background knowledge: Background knowledge may be used to
guide the discovery process and allow discovered patterns to be expressed in
concise terms and at different levels of abstraction.
Data mining query languages and ad hoc mining: Relational query languages
(such as SQL) allow users to pose ad hoc queries for data retrieval.
Presentation and visualization of data mining results: Discovered knowledge
should be expressed in high-level languages, visual representations, or other
expressive forms so that knowledge can be easily understood and directly usable
by humans.
Handling noisy or incomplete data: When mining data regularities, these objects
may confuse the process, causing the knowledge model constructed to overfit the
data.
Pattern evaluation--the interestingness problem: A data mining system can
uncover thousands of patterns. Many of the patterns discovered may be
uninteresting to the given user, representing common knowledge or lacking
novelty.

PERFORMANCE ISSUES

There can be performance-related issues such as follows

Efficiency and scalability of data mining algorithms

In order to effectively extract the information from huge amount of data in databases, data
mining algorithm must be efficient and scalable.

3
Parallel, distributed, and incremental mining algorithms

The factors such as huge size of databases, wide distribution of data, and complexity
of data mining methods motivate the development of parallel and distributed data mining
algorithms. These algorithms divide the data into partitions which is further processed in a
parallel fashion. Then the results from the partitions are merged. The incremental algorithms,
update databases without mining the data again from scratch.

DIVERSE DATA TYPES ISSUES

Handling of relational and complex types of data

The database may contain complex data objects, multimedia data objects, spatial data,
temporal data etc. It is not possible for one system to mine all these kind of data.

Mining information from heterogeneous databases and global information systems

The data is available at different data sources on LAN or WAN. These data source
may be structured, semi structured or unstructured. Therefore mining the knowledge from
them adds challenges to data mining.

CONCLUSION

One of the key issues raised by data mining technology is not a business or
technological one, but a social one. It is the issue of individual privacy. Data mining makes it
possible to analyze routine business transactions and glean a significant amount of
information about individuals buying habits and preferences.

Another issue is that of data integrity. Clearly, data analysis can only be as good as
the data that is being analyzed. A key implementation challenge is integrating conflicting or
redundant data from different sources. For example, a bank may maintain credit cards
accounts on several different databases. The addresses (or even the names) of a single
cardholder may be different in each. Software must translate data from one system to another
and select the address most recently entered.

4
 A hotly debated technical issue is whether it is better to set up a relational database
structure or a multidimensional one. In a relational structure, data is stored in tables,
permitting ad hoc queries. In a multidimensional structure, on the other hand, sets of cubes
are arranged in arrays, with subsets created according to category. While multidimensional
structures facilitate multidimensional data mining, relational structures thus far have
performed better in client/server environments. And, with the explosion of the Internet, the
world is becoming one big client/server environment.

Finally, there is the issue of cost. While system hardware costs have dropped
dramatically within the past five years, data mining and data warehousing tend to be self-
reinforcing. The more powerful the data mining queries, the greater the utility of the
information being gleaned from the data, and the greater the pressure to increase the amount
of data being collected and maintained, which increases the pressure for faster, more
powerful data mining queries. This increases pressure for larger, faster systems, which are
more expensive.

REFERENCES

http://www.tutorialspoint.com/data_mining/dm_issues.htm
http://www.anderson.ucla.edu/faculty/jason.frand/teacher/technologies/palace/issu
es.htm
http://searchcrm.techtarget.com/tip/Major-issues-in-data-mining