Introduction to algorithms

T H O M A S H. C O R M E N
C H A R L E S E. L E I S E R S O N
R O N A L D L. R I V E S T
C L I F F O R D STEIN

INTRODUCTION TO

ALGORITHMS
THIRD

EDITION

Introduction to Algorithms
Third Edition

Thomas H. Cormen
Charles E. Leiserson
Ronald L. Rivest
Clifford Stein

Introduction to Algorithms
Third Edition

The MIT Press
Cambridge, Massachusetts

London, Englandc 2009 Massachusetts Institute of Technology
All rights reserved. No part of this book may be reproduced in any form or by any electronic or mechanical means
(including photocopying, recording, or information storage and retrieval) without permission in writing from the
publisher.
For information about special quantity discounts, please email special sales@mitpress.mit.edu.
This book was setin Times Roman and Mathtime Pro 2 by the authors.
Printed and bound in the United States of America.
Library of Congress Cataloging-in-Publication Data
Introduction to algorithms / Thomas H. Cormen . . . [et al.].—3rd ed.
p. cm.
Includes bibliographical references and index.
ISBN 978-0-262-03384-8 (hardcover : alk. paper)—ISBN 978-0-262-53305-8 (pbk. : alk. paper)
1. Computer programming.2. Computer algorithms. I. Cormen, Thomas H.
QA76.6.I5858 2009
005.1—dc22
2009008593
10 9 8 7 6 5 4 3 2

Contents

Preface

xiii

I Foundations
Introduction

3

1

The Role of Algorithms in Computing 5
1.1 Algorithms 5
1.2 Algorithms as a technology 11

2

Getting Started 16
2.1 Insertion sort 16
2.2 Analyzing algorithms 23
2.3 Designing algorithms 29

3

Growth ofFunctions 43
3.1 Asymptotic notation 43
3.2 Standard notations and common functions

4

?
5

?

53

Divide-and-Conquer 65
4.1 The maximum-subarray problem 68
4.2 Strassen’s algorithm for matrix multiplication 75
4.3 The substitution method for solving recurrences 83
4.4 The recursion-tree method for solving recurrences 88
4.5 The master method for solving recurrences 93
4.6Proof of the master theorem 97
Probabilistic Analysis and Randomized Algorithms 114
5.1 The hiring problem 114
5.2 Indicator random variables 118
5.3 Randomized algorithms 122
5.4 Probabilistic analysis and further uses of indicator random variables
130

vi

Contents

II Sorting and Order Statistics
Introduction
6

7

8

9

147

Heapsort 151
6.1 Heaps 151
6.2 Maintainingthe heap property
6.3 Building a heap 156
6.4 The heapsort algorithm 159
6.5 Priority queues 162

154

Quicksort 170
7.1 Description of quicksort 170
7.2 Performance of quicksort 174
7.3 A randomized version of quicksort
7.4 Analysis of quicksort 180
Sorting in Linear Time 191
8.1 Lower bounds for sorting
8.2 Counting sort 194
8.3 Radix sort 197
8.4 Bucket sort 200

179

191Medians and Order Statistics 213
9.1 Minimum and maximum 214
9.2 Selection in expected linear time 215
9.3 Selection in worst-case linear time 220

III Data Structures
Introduction
10

11

?

229

Elementary Data Structures 232
10.1 Stacks and queues 232
10.2 Linked lists 236
10.3 Implementing pointers and objects
10.4 Representing rooted trees 246
Hash Tables 253
11.1Direct-address tables 254
11.2 Hash tables 256
11.3 Hash functions 262
11.4 Open addressing 269
11.5 Perfect hashing 277

241

Contents

12

?
13

14

vii

Binary Search Trees 286
12.1 What is a binary search tree? 286
12.2 Querying a binary search tree 289
12.3 Insertion and deletion 294
12.4 Randomly built binary search trees 299
Red-Black Trees 308
13.1 Properties of red-blacktrees
13.2 Rotations 312
13.3 Insertion 315
13.4 Deletion 323

308

Augmenting Data Structures 339
14.1 Dynamic order statistics 339
14.2 How to augment a data structure
14.3 Interval trees 348

345

IV Advanced Design and Analysis Techniques
Introduction

357

15

Dynamic Programming 359
15.1 Rod cutting 360
15.2 Matrix-chain multiplication 370
15.3 Elements of dynamic...