Editing Optimization Theory

<div style="background-color: #4B0082; color: #FFFFFF; padding: 20px; border-radius: 8px; margin-bottom: 15px;">
{{BloomIntro}}
Optimization Theory is the branch of mathematics that seeks to find the "Best" solution to a problem from a set of available alternatives. In the real world, we are always limited by time, money, or resources. Optimization is about maximizing what we want (like profit, speed, or accuracy) while minimizing what we don't want (like cost, fuel, or error). It is the math of '''Decision Making'''. From a GPS finding the fastest route to a factory scheduling its machines and an AI "learning" to recognize faces, optimization is the engine that drives efficiency in a complex world.
</div>

__TOC__

<div style="background-color: #000080; color: #FFFFFF; padding: 20px; border-radius: 8px; margin-bottom: 15px;">
== <span style="color: #FFFFFF;">Remembering</span> ==
* '''Optimization''' — The process of making something as effective or functional as possible.
* '''Objective Function''' — The mathematical function you want to maximize or minimize (e.g., $Profit = 10x + 5y$).
* '''Constraints''' — The limits or restrictions on the possible solutions (e.g., "I only have 40 hours of labor available").
* '''Feasible Region''' — The set of all possible points that satisfy all the constraints.
* '''Linear Programming (LP)''' — A method for solving optimization problems where both the objective and constraints are linear.
* '''Convex Optimization''' — A specific type of optimization where the "bowl" shape of the problem ensures that any local minimum is also the global minimum.
* '''Gradient Descent''' — An iterative algorithm used to find the minimum of a function by "stepping downhill."
* '''Local Minimum''' — The "Bottom" of a small area, which might not be the lowest point overall.
* '''Global Minimum''' — The absolute lowest point of the entire function.
* '''Heuristic''' — A "Rule of Thumb" used to find a "good enough" solution when the perfect one is too hard to find.
* '''Sensitivity Analysis''' — Studying how the optimal solution changes if the constraints change slightly.
* '''Stochastic Optimization''' — Optimization where some of the data is random or uncertain.
* '''Pareto Efficiency''' — A state where you cannot make one thing better without making something else worse.
</div>

<div style="background-color: #006400; color: #FFFFFF; padding: 20px; border-radius: 8px; margin-bottom: 15px;">
== <span style="color: #FFFFFF;">Understanding</span> ==
Optimization is like '''Finding the Bottom of a Foggy Valley'''.

'''1. The Three Components''':
* '''The Goal''': What are we trying to do? (Minimize distance).
* '''The Variables''': What can we change? (Which roads we take).
* '''The Limits''': What is stopping us? (One-way streets, traffic, fuel).

'''2. Convexity (The "Bowl" vs. The "Mountain Range")''':
* '''Convex''': The problem is a simple "U" shape. No matter where you start, if you walk downhill, you will reach the bottom.
* '''Non-Convex''': The problem has many small dips. You might think you've found the bottom, but the ''real'' bottom is on the other side of a hill. This is the biggest challenge in AI training.

'''3. Linear vs. Non-Linear''':
* '''Linear''': "If I buy 2x as much, it costs 2x as much." These problems are easy to solve using the '''Simplex Algorithm'''.
* '''Non-Linear''': Most of nature is non-linear. "If I push 2x as hard, the bridge might snap." These require more complex calculus.

'''The "No Free Lunch" Theorem''': This is a famous idea in optimization that says there is no single "Best" algorithm for every problem. An algorithm that is great for finding the fastest flight is likely terrible at training a neural network. You must match the "Tool" to the "Valley."
</div>

<div style="background-color: #8B0000; color: #FFFFFF; padding: 20px; border-radius: 8px; margin-bottom: 15px;">
== <span style="color: #FFFFFF;">Applying</span> ==
'''Modeling 'The Diet Problem' (Linear Programming):'''
<syntaxhighlight lang="python">
def simple_optimize_profit(hours_avail, price_a, price_b, time_per_a, time_per_b):
    """
    Max Profit = (n_a * price_a) + (n_b * price_b)
    Constraint: (n_a * time_per_a) + (n_b * time_per_b) <= hours_avail
    """
    # If we only make A
    count_a = hours_avail // time_per_a
    profit_a = count_a * price_a
    
    # If we only make B
    count_b = hours_avail // time_per_b
    profit_b = count_b * price_b
    
    if profit_a > profit_b:
        return f"Best: Make {count_a} of A. Profit: ${profit_a}"
    else:
        return f"Best: Make {count_b} of B. Profit: ${profit_b}"

# 40 hours, Product A ($50, 2h), Product B ($30, 1h)
print(simple_optimize_profit(40, 50, 30, 2, 1))
# Even though A is 'more expensive', B is better because 
# you can make more of it in the same time.
</syntaxhighlight>

; Optimization in Action
: '''Logistics (The Traveling Salesman)''' → Finding the shortest route to visit 100 cities. It is a "Hard" problem that powers UPS and FedEx.
: '''Portfolio Optimization''' → Choosing a mix of stocks to get the highest return for the lowest risk.
: '''Backpropagation''' → The optimization algorithm that trains AI by minimizing the "Error" between the AI's guess and the truth.
: '''Structural Engineering''' → Designing a bridge with the minimum amount of steel that can still support 100 tons.
</div>

<div style="background-color: #8B4500; color: #FFFFFF; padding: 20px; border-radius: 8px; margin-bottom: 15px;">
== <span style="color: #FFFFFF;">Analyzing</span> ==
{| class="wikitable"
|+ Gradient Descent vs. Brute Force
! Method !! How it works !! Advantage !! Disadvantage
|-
| Brute Force || Try every single option || Guaranteed to find the best || Too slow for big problems
|-
| Gradient Descent || Follow the slope down || Very fast / Works for AI || Can get stuck in a 'Local' dip
|-
| Genetic Algorithms || 'Evolve' solutions || Finds creative solutions || Not guaranteed to be optimal
|-
| Simplex Method || Jump between 'corners' || Perfect for linear problems || Hard to apply to physics
|}

'''The Concept of "Shadow Prices"''': In optimization, this is the "Value of one more." If you had 41 hours of labor instead of 40, how much more profit would you make? Analyzing these prices is how businesses decide whether it's worth it to hire a new employee or buy a new machine.
</div>

<div style="background-color: #483D8B; color: #FFFFFF; padding: 20px; border-radius: 8px; margin-bottom: 15px;">
== <span style="color: #FFFFFF;">Evaluating</span> ==
Evaluating an optimal solution:
# '''Feasibility''': Does the "Best" solution actually follow all the rules (e.g., did it use money we don't have)?
# '''Convergence''': Did the algorithm actually reach the bottom, or did it run out of time?
# '''Robustness''': If the price of fuel goes up by 1%, does our "Best" plan fall apart?
# '''Pareto Trade-offs''': If we have two goals (Speed and Cost), are we at the point where we can't improve one without hurting the other?
</div>

<div style="background-color: #2F4F4F; color: #FFFFFF; padding: 20px; border-radius: 8px; margin-bottom: 15px;">
== <span style="color: #FFFFFF;">Creating</span> ==
Future Frontiers:
# '''Quantum Optimization (QAOA)''': Using quantum computers to solve "Hard" optimization problems in seconds.
# '''Differentiable Programming''': Building software where every single line of code can be "Optimized" by a computer.
# '''Multi-Agent Optimization''': How thousands of self-driving cars can optimize their routes together without a central brain.
# '''Evolutionary Design''': Using optimization to "Grow" the shapes of airplanes and rocket engines that no human would ever think of.

[[Category:Mathematics]]
[[Category:Economics]]
[[Category:Computer Science]]
</div>