Yale

Industrial and Government Applications Track Poster

YALE: Rapid Prototyping for Complex Data Mining Tasks
Ingo Mierswa
Artificial Intelligence Unit Department of Computer Science University of Dortmund

Michael Wurst
Artificial Intelligence Unit Department of Computer Science University of Dortmund

Ralf Klinkenberg
Artificial Intelligence Unit Department of Computer Science University ofDortmund

ingo.mierswa@unidortmund.de

wurst@ls8.cs.unidortmund.de

ralf.klinkenberg@unidortmund.de

Martin Scholz
Artificial Intelligence Unit Department of Computer Science University of Dortmund

Timm Euler
Artificial Intelligence Unit Department of Computer Science University of Dortmund

scholz@ls8.cs.unidortmund.de ABSTRACT
KDD is a complex and demanding task. While a largenumber of methods has been established for numerous problems, many challenges remain to be solved. New tasks emerge requiring the development of new methods or processing schemes. Like in software development, the development of such solutions demands for careful analysis, specification, implementation, and testing. Rapid prototyping is an approach which allows crucial design decisions as early aspossible. A rapid prototyping system should support maximal re-use and innovative combinations of existing methods, as well as simple and quick integration of new ones. This paper describes Yale, a free open-source environment for KDD and machine learning. Yale provides a rich variety of methods which allows rapid prototyping for new applications and makes costly re-implementations unnecessary.Additionally, Yale offers extensive functionality for process evaluation and optimization which is a crucial property for any KDD rapid prototyping tool. Following the paradigm of visual programming eases the design of processing schemes. While the graphical user interface supports interactive design, the underlying XML representation enables automated applications after the prototyping phase. After adiscussion of the key concepts of Yale, we illustrate the advantages of rapid prototyping for KDD on case studies ranging from data pre-processing to result visualization. These case studies cover tasks like feature engineering, text mining, data stream mining and tracking drifting

timm.euler@unidortmund.de
concepts, ensemble methods and distributed data mining. This variety of applications isalso reflected in a broad user base, we counted more than 40,000 downloads during the last twelve months. Track: Industrial Track Categories and Subject Descriptors: I.5.2 [Computing Methodologies]: Pattern Recognition General Terms: Design, Experimentation Keywords: KDD system, rapid prototyping, multimedia mining, audio and text mining, data stream mining, data pre-processing, resultvisualization, distributed data mining, feature construction

1. INTRODUCTION
It is well known that knowledge discovery (KD) is a highly complex process. Like software development, it requires careful analysis, specification, implementation and testing. Prototyping plays an important role in this process. On the one hand, prototyping helps to identify adequate methods and optimal parameters. This enablesdevelopers to make crucial design decisions as early as possible in the knowledge discovery process. Costly redesign at later stages can be avoided. On the other hand, prototyping helps to control several risks. Most importantly, the performance of the envisioned system can be estimated beforehand. This gives the customer an impression of the final result and its limitations. It also helps to clarifymisunderstandings concerning the envisioned outcome. Another important aspect is to estimate computation time and cost of the final system. Especially for applications with tight constraints on these resources (e.g. real time systems), such an estimation is essential in order to decide to which extent knowledge discovery can be applied. A prototyping framework for knowledge discovery must meet...