Support Vector Machine

KNOWLEDGE DISCOVERY WITH SUPPORT VECTOR MACHINES

LUTZ HAMEL
University of Rhode Island

A JOHN WILEY & SONS, INC., PUBLICATION

KNOWLEDGE DISCOVERY WITH SUPPORT VECTOR MACHINES

KNOWLEDGE DISCOVERY WITH SUPPORT VECTOR MACHINES

LUTZ HAMEL
University of Rhode Island

A JOHN WILEY & SONS, INC., PUBLICATION

Copyright © 2009 by John Wiley& Sons, Inc. All rights reserved.Published by John Wiley & Sons, Inc., Hoboken, New Jersey. Published simultaneously in Canada. No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning, or otherwise, except as permitted under Section 107 or 108 of the 1976 United States Copyright Act, without either the priorwritten permission of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, (978) 750-8400, fax (978) 750-4470, or on the web at www.copyright.com. Requests to the Publisher for permission should be addressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ07030, (201) 748-6011, fax (201) 748-6008, or online at http://www.wiley.com/go/permission. Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their best efforts in preparing this book, they make no representations or warranties with respect to the accuracy or completeness of the contents of this book and specifically disclaim any implied warranties of merchantabilityor fitness for a particular purpose. No warranty may be created or extended by sales representatives or written sales materials. The advice and strategies contained herein may not be suitable for your situation. You should consult with a professional where appropriate. Neither the publisher nor author shall be liable for any loss of profit or any other commercial damages, including but not limited tospecial, incidental, consequential, or other damages. For general information on our other products and services or for technical support, please contact our Customer Care Department within the United States at (800) 762-2974, outside the United States at (317) 572-3993 or fax (317) 572-4002. Wiley also publishes its books in a variety of electronic formats. Some content that appears in print maynot be available in electronic formats. For more information about Wiley products, visit our web site at www.wiley.com. Library of Congress Cataloging-in-Publication Data: Hamel, Lutz. Knowledge discovery with support vector machines / Lutz Hamel. p. cm. — (Wiley series on methods and applications in data mining) Includes bibliographical references and index. ISBN 978-0-470-37192-3 (cloth) 1.Support vector machines. 2. Data mining. 3. Machine learning. 4. Computer algorithms. I. Title. Q325.5.H38 2009 005.1–dc22 2009011948 Printed in the United States of America 10 9 8 7 6 5 4 3 2 1

To Natalie, Enzo, and Oliver and in memory of Klaus

CONTENTS

PREFACE PART I
1 WHAT IS KNOWLEDGE DISCOVERY? 1.1 1.2 1.3 1.4 Machine Learning Structure of the Universe X Inductive Learning ModelRepresentations Exercises Bibliographic Notes

xiii
1 3 4 6 8 9 11 11 13 13 14 17 20 23 26
vii

2 KNOWLEDGE DISCOVERY ENVIRONMENTS 2.1 Computational Aspects of Knowledge Discovery 2.1.1 Data Access 2.1.2 Visualization 2.1.3 Data Manipulation 2.1.4 Model Building and Evaluation 2.1.5 Model Deployment

viii

CONTENTS

2.2 Other Tool Sets Exercises Bibliographic Notes 3 DESCRIBING DATAMATHEMATICALLY 3.1 From Data Sets to Vector Spaces 3.1.1 Vectors 3.1.2 Vector Spaces 3.2 The Dot Product as a Similarity Score 3.3 Lines, Planes, and Hyperplanes Exercises Bibliographic Notes 4 LINEAR DECISION SURFACES AND FUNCTIONS 4.1 From Data Sets to Decision Functions 4.1.1 Linear Decision Surfaces Through the Origin 4.1.2 Decision Surfaces with an Offset Term 4.2 Simple Learning Algorithm 4.3...