The field of Knowledge Discovery and Data Mining has grown rapidly in recent years. Massive data sets have driven research, applications, and tool development in business, science, government, and academia. The continued growth in data collection in all of these areas ensures that the fundamental problem ... KDD

The term "data mining" is often used incorrectly to apply to a variety of unrelated processes. In many cases, applications may claim to perform "data mining" by automating the creation of charts or graphs with historic trends and analysis. more

Data Mining Software

Definition: Data mining is the term used to describe the technique of automatically analysing large volumes of data in order to identify patterns and associations within that data.


Data Mining Software

Enterprises succeed or fail on the strength of the information they possess. In order to successfully design, manufacture and market products and services an enterprise needs accurate data to ensure that the job is managed correctly – and to ensure that the strategies and objectives of the enterprise align with the realities of the marketplace,

However, simply possessing large volumes of data is useless without the means to draw valuable conclusions from it. Without methods of managing and analysing data the information is nothing more than a drain on the resources of an enterprise.

Applications of Data Mining

The 1990s saw great leaps in the development of data collection, storage and processing techniques and technologies. These developments allowed enterprises to gather and compile vast amounts of data related to sales, customer behaviour and the marketplace. However, all too often technology limitations meant that these data resources were not adequately exploited.

In recent years there have been a renewed focus on the deep analysis of business data, encouraging enterprises to look past basic data and search for hidden patterns and associations to aid in their business. A wide range of industries now use data mining software to seek out and exploit unexpected trends.

- Customer Retention

The vast majority of medium to large enterprises collect data about their customers for the purposes of database marketing. However, only through the use of data mining can this data become useful information. Through deep analysis of the purchasing patterns of past customers it is possible to develop a demographic model of those customers who can be expected to re-purchase in the future. By directing advertising funds only towards those customers an enterprise can increase return on investment (ROI) of the advertising budget and build long term client-customer relationships.

- Cross Selling Opportunities

In addition to analysing data about previous customers, enterprises can also data mine customer databases purchased from other organisations in the hope of identifying cross selling opportunities.

Data Mining SoftwareFor example, charity organisations such as Oxfam retain data about their members and contributors. If another charity can get hold of that data it can be analysed to identify potential new contributors.

The possibilities are more diverse for commercial enterprises. Databases of purchasers of personal computers and games can be a gold mine for the enterprises that develop software and games for those systems. By analysing data and grouping customers by demographics such as age and gender, it is possible to build highly targeted marketing databases of customers likely to purchase software and games from various genres – shoot-em-ups for teens, office software for students and adults and platform games for children of all ages. The possibilities are endless.

- Identify Market Trends

Perhaps most importantly for enterprises developing new products is the data mining of economic and industry-specific data to identify market trends. This sort of analysis is vital during the conception process of any new product, as it allows enterprises to predict the demand for a product before it has even left the drawing board.

The identification of market trends can also help enterprises innovate new products that will ride the wave at the forefront of their industry. By gaining insight into the way the wind is blowing, enterprises can avoid sinking capital into products that are yesterday’s news – something that is especially important in high-tech industries.

Pitfalls of Data Mining

While data mining may seem like the perfect data analysis solution – boiling down massive volumes of data into valuable nuggets of information in the blink of an eye – enterprises would do well to reserve their trust in the results.

The dark side of data mining is known as data dredging. Data dredging is the pejorative term used to describe the use of data mining to find any possible patterns in the data. Often the result of over-eager or poorly implemented data mining techniques, data dredging may show patterns in the data that are essentially meaningless.

For example, data dredging of a sales database may show that sales of chocolate to blind people rose by 5% over the quarter. While this may indeed be the case, it is not grounds to re-allocate an advertising budget to appeal to the blind and partially sighted demographic. Instead, the conclusions should be retested using a separate data set. If the pattern repeats itself on retesting then it may be valid. If not, it is probably simply an anomaly.

Further information regarding data mining can be found at the Lawrence Livermore National Laboratory.