Outline

• Asymptotic notations and their relationships
• Asymptotic dominance
• The power of efficient algorithms
• Logarithms in algorithm analysis
• Binary search
• Basic algorithm analysis
• Recursive algorithms
• Limits of computation
• Complexity of a problem

Objective

• To be able to prove relationships among asymptotic notations
• Can apply the technique to analyze any algorithms
• Explain the logarithm terms that appear in the analysis
• Can analyze iterative and recursive algorithms.
• Can show the complexity class of any given problems

Summary

Common asymptotic class

Implications of Asymptotic Dominance

• Exponential algorithms are hopeless for anything beyond very small inputs
• Quadratic algorithms are hopeless beyond about one million
• O($n \log n$) is possible to about one billion
• O($\log n$) never struggles
• Logarithmic time algorithms grow remarkably slowly
• Logs crop up often in algorithm analysis

Logarithm in Algorithm analysis

• Note that $$b^x = y \mbox{ means } x = \log_b y$$ Or equivalently, $$b^{\log_b y} = y$$ Logarithms arise whenever we repeatedly halve something
• Recall: $\log_a (x y) = \log_a (x) + \log_a (y)$ and $\log_b (a)$ = $\displaystyle\frac{\log_c a}{\log_c b}$
• The base doesn’t matter in asymptotic analysis
• Changing from base $a$ to $c$ is just multiplying by a constant: $$\log_a b = K log_c b \mbox{ where } K = \frac{1}{\log_c a}$$

Tower of Hanoi analysis

• No standard method for solving recurrences
• Can learn some tricks
• In general, make informed guess at solution and prove using induction
• An intuitive approach is a recursion tree.

Computational class and feasiblity

Q: Where do we draw the line? What is an “efficient” algorithm?
Is there a limit to what we can consider feasible?

Polynomial time algorithm

• This decision is somewhat arbitrary
• We equate an efficient algorithm with one which runs in polynomial time $$T(n) = O(p(n)) \mbox{ and } \exists k \in \mathbb{N}, p(n) = O(n^k)$$
• We call an algorithm with running time $T(n)$ is a polynomial-time algorithm.

Complexity of a problem

• We’ve been thinking about the complexity of an algorithm
  Q: What might we mean by the complexity of a problem?
  A: Lowest worst-case complexity of any algorithm that can solve it
• Arguing exact complexity is sometimes difficult
• We may only be able to give lower bounds (examples later)
• We can now talk about “polynomial-time solvable” problems
  ...meaning: there is a polynomial-time algorithm that can solve the problem
• For many important problems there is no known efficient algorithm.

Examples of hard problems

• The shortest path problem for a robot arm we saw in Lecture 1 is commonly known as the “Travelling Salesman Problem”
• Huge implications for logistics if it could be solved
• Knapsack problem: given a set of items, each with a weight and value, can a value of at least $V$ be achieved without exceeding the weight $W$?
• These problems are easy to explain, easy to verify that a solution is correct, but extremely difficult to solve
• For anything but trivial problems, computation time is astronomical.
• There is a class of such problems which are all equivalent (solve one = solve all)
• They are known as NP-complete problems.

NP-complete problems

• NP stands for Nondeterministic Polynomial time
• NP-complete problems have two properties:
  • Any given solution to the problem can be verified in polynomial time
  • If the problem can be solved in polynomial time, so can every problem in NP
• At present, establishing your problem is NP-complete means don’t bother trying to solve it exactly
• One of the great problems in computer science is to establish whether a polynomial time algorithm can be found to solve an NP-complete problem
• Although it is thought unlikely that P = NP (since no-one has found such an algorithm yet) it also not been proven that P ≠ NP

P = NP?

Solve “P = NP?” and EARN $\$1,000,000$

• This is such an important problem that the Clay Mathematics Institute has offered a one million dollar prize for its solution.
  Show that P = NP and EARN $\$7,000,000$
• Some people think that if P = NP it may be possible for computers to “find a formal proof of any theorem which has a proof of a reasonable length”
• It is therefore likely that you could use a computer to solve the other 6 CMI prize problems and claim all of the prize money
• Get working on it!

Conclusion

• We now have a set of tools to compare algorithms:
  • We know how to go from pseudocode to algorithm complexity
  • Logarithmic time algorithms arise from divide-and-conquer - very efficient and can be powerful
  • We know how to compare complexities and when to say “too slow!”

Comments and Suggestions

• Assistant Professor Krung Sinapiromsaran
• Web: http://pioneer.netserv.chula.ac.th/~skrung
• Email: Krung.S@chula.ac.th