Abstract Datatypes and Data Structures: Stacks, Queues, ...

(抽象データ型とデータ構造、スタック、キューなど)

Data Structures and Algorithms

4th lecture, October 12, 2017

http://www.sw.it.aoyama.ac.jp/2017/DA/lecture4.html

Martin J. Dürst

Today's Schedule

Summary and homework of last lecture
Polynomial vs. exponential time
Finding the (asymptotic) time complexity of an algorithm
Recurrence relations
Abstract Data Types
- Stack
- Queue

Summary of Last Lecture

The asymptotic growth (order of growth) of a function and the time (and space) complexity of an algorithm can be expressed with the Big-O/Ω/Θ notation:

O(g(n)): Set of functions with lower or same order of growth as g(n)
Ω(g(n)): Set of functions with larger or same order of growth as g(n)
Θ(g(n)): Set of functions with same order of growth as g(n)

f(n)∈O(g(n)) ⇔ ∃c>0: ∃n₀≥0: ∀n≥n₀: f(n)≤c·g(n)

The order of growth of a function can be found by:

Looking for appropriate c and n₀
Calculating lim_n→∞(f(n)/g(n))
Simplification (e.g. O(5n²+300n+200000) → O(n²))

When using Big-O notation, always try to simplify g() as much as possible.

Last Week's Homework

(no need to submit)

Review this lecture's material every day!

On the Web, find algorithms with time complexity O(1), O(log n), O(n), O(n log n), O(n²), O(n³), O(2ⁿ), O(n!), and so on.

Frequent Orders

O(1): Simple formulæ (e.g. interest calculation), initialization

O(log n) (logarithmic order/time): binary search, other "divide and conquer" algorithms

O(n) (linear order/time): proportional to size of data, checking all data items once (or a finite number of times)

O(n log n): Many sorting algorithms, other "divide and conquer" algorithms

O(n²) (quadratic order/time), O(n³) (cubic order/time): Considering all/most combinations of 2 or 3 data items

O(2ⁿ): Considering all/most subsets of data items

O(n!): Considering all/most permutations of data items

Polynomial versus Exponential Growth

Example:

1.1ⁿ ≶ n²⁰

log(1.1)·n ≶ log(n)·20

n/log₁₀(n) ≶ 20/log₁₀(1.1) ≊483.2

n₀ ≊ 1541

Conclusion: For a, b > 1, aⁿ will always eventually grow faster than n^b

(n^b is polynominal, aⁿ is exponential)

The Importance of Polynomial Time

What can be called a "realistic" time complexity depends on the problem
In general:
- Polynomial time is realistic
- Exponential time is unrealistic

[We will discuss this in more detail in lecture 14]

Finding the (Asymptotic) Time Complexity of an Algorithm

Find/define the variables that determine the size of the input (e.g. n)
Find the basic operations (steps) in the algorithm that are most frequently executed
Express the total number of basic operations (steps) using summation or a recurrence relation
Determine the time complexity expressed with big-O notation

Simplifications possible for big-O notation can be applied early.
Example: Because constant factors are irrelevant in big-O notation, they can be eliminated when counting steps.

How to Define Input Size Variables

In many cases, the input size is the number of data items (examples: search, sort)
For matrices, ..., often the number of rows or columns is used (matrices of size n × n or n × m)
In some cases, the size of individual data items has to be considered
Examples: Size in bits of integers with unlimited precision; length of variable-length strings, ...
Sometimes, there are two or more kinds of data, with different size
Example: String matching (text size n and pattern size m, lecture 11)

How to Identify the Most Frequent Basic Operations

Usually inside a loop (especially inside multiple loops)
If there are several independent loops, check all of them
If the number of operations depends on the values in the input, check the worst case
Example: For search, time if value is not found
When methonds/functions are called, consider the content of the function

Caution: Some methods/functions may hide complexity (e.g. Ruby sort, ...)

Counting Basic Operations using Summation

Example program:

for (i=0; i<n; i++)
    for (j=i; j<n; j++)
        sum += i*j;

Most frequent operation: Addition or multiplication in last line (inner loop)
Expressing the number of operations as a sum:
∑_i=0^n-1 ∑_j=i^n-1 1
Evaluating the sum:
∑_i=0^n-1 ∑_j=i^n-1 1 = ∑_i=0^n-1 n-i =
= n + n-1 + n-2 + ... + 2 + 1 = n · (n+1) / 2
Asymptotic time complexity:
n · (n+1) / 2 = 0.5 n² + 0.5 n∈ O(n²)

Counting Basic Operations using Recurrence Relations

Example program (recursive version of binary search):

def binsearch(array, low, high, key)
  middle = (high+low)/2
  if low>=high
    array[low]==key ? low : nil
  elsif key>array[middle]
    binsearch(array, middle+1, high, key)
  else
    binsearch(array, low, middle, key)
  end
end

Expressing the number of operations as a recurrence:
B(1) = 1
B(n) = B(⌈n/2⌉) + 1
(⌈⌉: ceiling function)

Recurrence Relations

A recurrence (relation) is a recursive definition of a mathematical function
There are several ways to solve recurrences
One way to solve a recurrence is to discover a pattern by repeated substitution:
B(n) = B(⌈n/2⌉) + 1 = B(⌈⌈n/2⌉/2⌉) + 1 + 1 = B(⌈n/2²⌉) + 2 =
= B(⌈n/2³⌉) + 3 = B(⌈n/2^k⌉) + k
Using B(1) = 1:
⌈n/2^k⌉ = 1 ⇒ 1 ≥ n/2^k (>1/2) ⇒ 2^k ≥ n (> 2^k-1) ⇒ k ≥ log₂ n (> k-1) ⇒ k = ⌈log₂ n⌉
B(n) = 1 + ⌈log₂ n⌉ ∈ O(log n)
The asymptotic time complexity of binary search is O(log n)

Comparing the Execution Time of Algorithms

(from previous lectures)

Possible questions:

~~How many seconds faster is binary search when compared to linear search?~~
~~How many times faster is binary search when compared to linear search?~~
What is the order [of growth of the execution time] of linear search and binary search?
Linear search is O(n), binary search is O(log n).

Conclusion: Expressing time complexity as O() allows to evaluate the essence of an algorithm, ignoring hardware and implementation differences.

Abstract Data Type (ADT)

Combination of data with functions operating on data
The data can only be accessed/changed using the functions (encapsulation)
Goals
- Data integrity (example: birthday and age)
- Modularization of big software projects
Related to type theory
Often implemented by objects in object-oriended programming languages
- Type → class
- Function → member function/method

Typical Examples of Abstract Data Types

Stack
Queue
Linear list
Dictionary
Caution: A dictionary ADT is not exactly the same as a dictionary book
Priority queue

Stack

Principle:: last-in-first-out (LIFO)
General example:: Stack of trays in cafeteria
Example from IT:: Function stack (local variables, return address, ...)
Main methods:: new, add/push, delete/pop
Other methods:: empty? (check whether the stack is empty or not)
top (return the topmost element without removing it from the stack)

Axioms for Stacks

It is possible to define a stack using the following four axioms:

Stack.new.empty? ↔ true
s.push(e).empty? ↔ false
s.push(e).top ↔ e
s.push(e).pop ↔ s (here, pop returns the new stack, not the top element)

(s is any arbitrary stack, e is any arbitrary data item)

Axioms can define a contract between implementation and users

Queue

Principle:: first-in-first-out (FIFO)
General example:: Queue in cafeteria waiting for food
Example from IT:: Queue of processes waiting for execution
Main methods:: add/enqueue, remove/delete/dequeue
Explain the meaning of GIGO: Garbage in, garbage out.

Comparing ADTs

Implementation: 4ADTs.rb


ADT	stack		queue
Implemented as	`Array`	`LinearList`	`Array`	`LinearList`
create	`O`(`n`)	`O`(1)	`O`(`n`)	`O`(1)
add	`O`(`n`) or `O`(1)	`O`(1) or `O`(`n`)	`O`(`n`)	`O`(1)
delete	`O`(1) or `O`(`n`)	`O`(`n`) or `O`(1)	`O`(`n`)	`O`(1)
`empty?`	`O`(1)	`O`(1)	`O`(1)	`O`(1)
length	`O`(1)	`O`(`n`)	`O`(1)	`O`(`n`)

Summary

The order (of growth)/(asymptotic) time complexity of an algorithm can be calculated from the number of the most frequent basic operations
Calculation can use a summation or a recurrence (relation)
The big-O notation compactly express the inherent efficiency of an algorithm
An abstract data type (ADT) combines data and the operations on this data
Stack and queue are typical examples of ADTs

Homework

(no need to submit)

Order the following orders of growth, and explain the reason for your order:
O(n²), O(n!), O(n log log n), O(n log n), O(20ⁿ)
Write a simple program that uses the classes in 4ADTs.rb.
Use this program to compare the implementations.
Hint: Use the second part of 2search.rb as an example.
Implement the priority queue ADT (use Ruby or any other programming language)
A priority queue keeps a priority (e.g. integer) for each data item.
In the simplest case, the only data is the priority.
The items with the highest priority leave the queue first.
Your implementation can use an array or a linked list or any other data structure.

Report: Manual Sorting

Deadline: November 1, 2017 (Wednesday), 19:00.

Where to submit: Box in front of room O-529 (building O, 5th floor)

Format:

A4, double-sided 4 pages (2 sheets of paper, stapled in upper left corner; NO cover page)
Easily readable handwriting (NO printouts)
Name (kanji and kana), student number, course name and report name at the top of the front page

Problem: Propose and describe an algorithm for manual sorting, for the following two cases:

One person sorts 4000 pages
20 people together sort 60000 pages

Each page is a sheet of paper of size A4, where a 10-digit number is printed in big letters.

The goal is to sort the pages by increasing number. There is no knowledge about the distribution of the numbers.

You can use the same algorithm for both cases, or a different algorithm.

Details:

Describe the algorithm(s) in detail, so that e.g. your friends who don't understand computers can execute them.
Describe the equipment/space that you need.
Calculate the overall time needed for each case.
Analyse the time complexity (O()) of the algorithm(s).
Comment on the relationship to other algorithms you know, and on the special needs of manual (as opposed to computer) execution.
If you use any Web pages, books, ..., list them as references at the end of your report
Caution: Use IRIs (e.g. http://ja.wikipedia.org/wiki/情報), not URLs (e.g. http://ja.wikipedia.org/wiki/%E6%83%85%E5%A0%B1)

Glossary

polynomial growth: 多項式増加
exponential growth: 指数的増加
integers with unlimited precision: 非固定長整数
summation: 総和
recurrence (relation): 漸化式
ceiling function: 天井関数
substitution: 置換
abstract data type: 抽象データ型
encapsulation: カプセル化
data integrity: データの完全性
modularization: モジュール化
type theory: 型理論
object-oriended: オブジェクト指向 (形容詞)
type: 型
class: クラス
member function: メンバ関数
method: メソッド
stack: スタック
cafeteria: 食堂
axiom: 公理
queue: 待ち行列、キュー
ring buffer: リングバッファ
priority queue: 順位キュー、優先順位キュー、優先順位付き待ち行列