Optimization: In Optimization we either minimize or maximize objective functions / cost function.
No Free Lunch Theorem for Optimization
No Free Lunch Theorem for Optimization
According to No Free Lunch Theorem "There is no universal better algorithm exist that can solve all types of optimization problems".
Today, Metaheuristic Optimization Algorithms are used in different areas to solve complex real work optimization problems. For example in Industrial Areas, Operation Research, Medical Field, Engineering design and other as you can see below:
History of Metaheuristic Optimization Algorithms:
Genetic Algorithms (G.A.) - 1960's - 1970's
Simulated Annealing (S.A.) - 1983
Tabu Search (T.S.) - 1986
Ant Colony Optimization Algorithm - 1992
Particle Swarm Optimization Algorithm - 1995
Differential Evolution (D.E.) -1997
Harmony Search (H.S.) - 2001
Honey Bee Algorithm (H.B.A.) - 2004
Artificial Bee Colony (A.B.C.) - 2005
...
Battle Royal Optimization Algorithm (B.R.O.A.) - 2020
In 1997, D.H. Wolpher and W. G. Macready published No Free Lunch Theorem for optimization. According to No Free Lunch Theorem "There is no universal better algorithm exist that can solve all types of optimization problems". It is always hard to find a universal better way to solve almost all problems. There is no metaheuristic algorithm is best for solving all optimization problems.
One algorithm alone can't solve all types of optimization problems. There may be another algorithm that can provide better solutions to the problem that are not solved by first algorithm.
Suppose 2 Algorithm's, Algorithm X and Algorithm Y.
If Algorithm X performance is better than Algorithm Y for some objective function / optimization functions. Then Algorithm Y Will outperform Algorithm X for other optimization function/ objective functions.
Minimization Types: Constrained and Unconstrained Minimization, Linear Programming, Scalar Minimization Quadratic Programming.
Maximization Types:
Minimax
Semi-infinite
Minimization
Classes of Optimization Problems:
1. Linear vs Nonlinear
2. Single vs Multiple Objective
3. Discrete vs Continuous Design
4. Constrained vs Unconstrained Optimization
Discrete Optimization: When design variables are restricted to take on only given prescribed set of real values is known as Discrete Optimization.
Single Objective Optimization: In single objective optimization technique, we deal with only one objective function.
Multi-Objective Optimization: In multi-objective optimization technique, we deal with two or more than two objective functions. [we optimize set of objectives]
Approaches to Optimization:
1.) Analytical
2.) Numerical
3.) Graphical
4.) Experimental
Problem Formulation for Optimization Problem:
minimize ( x) f(x) // f(x) represents the Objective Function.
Subject to
g(x)<= 0 // g(x) represent the vector of Inequality Constraints
h (x) = 0 // h(x) represent the vector of Equality Constraints
LB <= x <= UB // x represent design variable, LB: Lower Bound, UB: Upper Bound
Optimization Problems are very complex as they provide more than one local optimal solutions. Various Categories for Optimization Problems are:
Constrained or Unconstrained Optimization Problems.
Discrete or Continuous Optimization Problems.
Static or Dynamic Optimization Problems.
Single or Multi-Objective Optimization Problems.
Metaheuristic Optimization Algorithms are widely accepted by researchers to solve various complex real life optimization problems. Metaheuristic Algorithms provide accurate, feasible solutions within time. Today, Metaheuristic Algorithms are used in different areas to solve optimization problems such as:
In Industrial Areas.
In Medical Field.
In Artificial Intelligence.
Mathematical Programming.
Operation Research.
Soft Computing.
To solve engineering design problems.
In Economy and other.
Metaheuristic Optimization Algorithms are easy to implement. Metaheuristics are divided into two categories:
Single Solution Based Metaheuristic Algorithms: In single solution based algorithms searching process starts with one single solution.
Multiple Solution Based Metaheuristic Algorithms: In multiple solution based algorithms searching process starts with a set of solutions (i.e., Initial Population). This is also known as Population based Metaheuristic Optimization.
Steps involved in Population Based Metaheuristic Algorithms:
Step 1: Randomly generate initial population (i.e., set of multiple solutions).
Step 2: Individuals / agents can share information with each other in the search space and avoid local optimal solutions.
Step 3: Exploration and Exploitations Phases.
Step 4: In the end when stopping criteria met display best / optimal / feasible solution obtained.
PARTICLE SWARM OPTIMIZATION ALGORITHM NUMERICAL EXAMPLE PSO is a computational method that Optimizes a problem. It is a Population-based stochastic search algorithm. PSO is inspired by the Social Behavior of Birds flocking. n Particle Swarm Optimization the solution of the problem is represented using Particles. [Flocking birds are replaced with particles for algorithm simplicity]. Objective Function is used for the performance evaluation for each particle / agent in the current population. PSO solved problems by having a Population (called Swarms) of Candidate Solutions (Particles). Local and global optimal solutions are used to update particle position in each iteration. Particle Swarm Optimization (PSO) Algorithm step-by-step explanation with Numerical Example and source code implementation. - PART 2 [Example 2] 1.) Initialize Population [Current Iteration (t) = 0] Population Size = 4; 𝑥𝑖 : (i = 1,2,3,4) and (t = 0) 𝑥1 =1.3; 𝑥2=4.3; 𝑥3=0.4; 𝑥4=−1.2 2.) Fitness Function u...
Cuckoo Search Algorithm - Metaheuristic Optimization Algorithm What is Cuckoo Search Algorithm? Cuckoo Search Algorithm is a Meta-Heuristic Algorithm. Cuckoo Search Algorithm is inspired by some Cuckoo species laying their eggs in the nest of other species of birds. In this algorithm, we have 2 bird Species. 1.) Cuckoo birds 2.) Host Birds (Other Species) What if Host Bird discovered cuckoo eggs? Cuckoo eggs can be found by Host Bird. Host bird discovers cuckoos egg with Probability of discovery of alien eggs. If Host Bird Discovered Cuckoo Bird Eggs. The host bird can throw the egg away. Abandon the nest and build a completely new nest. Mathematically, Each egg represent a solution and it is stored in the host bird nest. In this algorithm Artificial Cuckoo Birds are used. Artificial Cuckoo can lay one egg at a time. We will replace New and better solutions with less fit solutions. It means eggs that are more similar to host bird has opportunity to de...
Particle Swarm Optimization (PSO) is a p opulation-based stochastic search algorithm. PSO is inspired by the Social Behavior of Birds flocking. PSO is a computational method that Optimizes a problem. PSO searches for Optima by updating generations. It is popular is an intelligent metaheuristic algorithm. In Particle Swarm Optimization the solution of the problem is represented using Particles. [Flocking birds are replaced with particles for algorithm simplicity]. Objective Function is used for the performance evaluation for each particle / agent in the current population. After a number of iterations agents / particles will find out optimal solution in the search space. Q. What is PSO? A. PSO is a computational method that Optimizes a problem. Q. How PSO will optimize? A. By Improving a Candidate Solution. Q. How PSO Solve Problems? A. PSO solved problems by having a Population (called Swarms) of Candidate Solutions (Particles). Local and global optimal solutions are used to ...
Particle swarm optimization (PSO) What is meant by PSO? PSO is a computational method that Optimizes a problem. It is a Population-based stochastic search algorithm. PSO is inspired by the Social Behavior of Birds flocking. n Particle Swarm Optimization the solution of the problem is represented using Particles. [Flocking birds are replaced with particles for algorithm simplicity]. Objective Function is used for the performance evaluation for each particle / agent in the current population. PSO solved problems by having a Population (called Swarms) of Candidate Solutions (Particles). Local and global optimal solutions are used to update particle position in each iteration. How PSO will optimize? By Improving a Candidate Solution. How PSO Solve Problems? PSO solved problems by having a Population (called Swarms) of Candidate Solutions (Particles). The population of Candidate Solutions (i.e., Particles). What is Search Space in PSO? It is the range in which the algorithm computes th...
Local Binary Pattern Introduction to Local Binary Pattern (LBP) Q. What is Digital Image? A. Digital images are collections of pixels or numbers ( range from 0 to 255). Q. What is Pixel? A. Pixel is the smallest element of any digital image. Pixel can be categorized as Dark Pixel and Bright Pixel. Dark pixels contain low pixel values and bright pixels contain high pixel values. Q. Explain Local Binary Pattern (LBP)? A. Local binary pattern is a popular technique used for image processing. We can use the local binary pattern for face detection and face recognition. Q. What is LBP Operator? A. LBP operator is an image operator. We can transform images into arrays using the LBP operator. Q. How LBP values are computed? A. LBP works in 3x3 (it contain a 9-pixel value ). Local binary pattern looks at nine pixels at a time. Using each 3x3 window in the digital image, we can extract an LBP code. Q. How to Obtain LBP operator value? A. LBP operator values can be obtained by ...
Grey Wolf Optimization Algorithm (GWO) Grey Wolf Optimization Grey Wolf Optimization Algorithm is a metaheuristic proposed by Mirjaliali Mohammad and Lewis, 2014. Grey Wolf Optimizer is inspired by the social hierarchy and the hunting technique of Grey Wolves. What is Metaheuristic? Metaheuristic means a High-level problem-independent algorithmic framework (develop optimization algorithms). Metaheuristic algorithms find the best solution out of all possible solutions of optimization. Who are the Grey Wolves? Wolf (Animal): Wolf Lived in a highly organized pack. Also known as Gray wolf or Grey Wolf, is a large canine. Wolf Speed is 50-60 km/h. Their Lifespan is 6-8 years (in the wild). Scientific Name: Canis Lupus. Family: Canidae (Biological family of dog-like carnivorans). Grey Wolves lived in a highly organized pack. The average pack size ranges from 5-12. 4 different ranks of wolves in a pack: Alpha Wolf, Beta Wolf, Delta Wolf, and Omega Wolf. How Grey Wolf Optimiza...
There are about 1000 species of Bats. Bat Algorithm is based on the echolocation behavior of Micro Bats with varying pulse rates of emission and loudness. All bats use echolocation to sense distance and background barriers. Microbats are small to medium-sized flying mammals. Micro Bats used a Sonar that is known as Echolocation to detect their prey. Bats fly randomly with the velocity at the position with a fixed frequency and loudness for prey. Q. Whats is Frequency? A. Frequency is the number of waves that pass a fixed point in unit time. Wavelength is the minimum distance between two nearest particles which are in the same phase. Here, Sound waves are used by microbats to detect prey. Q. What is Position? A. A place where something or someone is located. Q. What is Velocity? A. Speed of something in a given direction. Q. What is loudness. A. Loudness refers to how soft or loud sound seems to listeners. Q. What is pulse rate? ...
Grey Wolf Optimization Algorithm Numerical Example Grey Wolf Optimization Algorithm Steps 1.) Initialize Grey Wolf Population. 2.) Initialize a, A, and C. 3.) Calculate the fitness of each search agent. 4.) 𝑿_𝜶 = best search agent 5.) 𝑿_𝜷 = second-best search agent 6.) 𝑿_𝜹 = third best search agent. 7.) while (t<Max number of iteration) 8.) For each search agent update the position of the current search agent by the above equations end for 9.) update a, A, and C 10.) Calculate the fitness of all search agents. 11.) update 𝑿_𝜶, 𝑿_𝜷, 𝑿_𝜹 12.) t = t+1 end while 13.) return 𝑿_𝜶 Grey Wolf Optimization Algorithm Numerical Example STEP 1. Initialize the Grey wolf Population [Initial Position for each Search Agent] 𝒙_(𝒊 ) (i = 1,2,3,…n) n = 6 // Number of Search Agents [ -100, 100] // Range Initial Wolf Position 3.2228 4.1553 -3.8197 4.2330 ...
Whale Optimization Algorithm Code Implementation Whale Optimization Algorithm Code Files function obj_fun(test_fun) switch test_fun case 'F1' x = -100:2:100; y=x; case 'F2' x = -10:2:10; y=x; end end function [LB,UB,D,FitFun]=test_fun_info(C) switch C case 'F1' FitFun = @F1; LB = -100; UB = 100; D = 30; case 'F2' FitFun = @F2; LB = -10; UB = 10; D = 30; end % F1 Test Function function r = F1(x) r = sum(x.^2); end % F2 Test Function function r = F2(x) r = sum(abs(x))+prod(abs(x)); end end function Position = initialize(Pop_Size,D,UB,LB) SS_Bo...
Comments
Post a Comment