Exploration / Exploitation

Reinforcement Learning is one of three main approaches for machine learning and can be described as follows: An autonomous agent observes the environment and performs actions to reach a pre-defined goal. A reward function gives feedback to the agent according to how close it was to reach the desired goal; otherwise, the agent has no information on which actions lead to the largest reward. The agent tries to maximize its gained reward in each learning iteration. And in every learning iteration the agent is caught in the exploration / exploitation dilemma: It could either exploit the already gained knowledge of the environment to safely receive the highest reward that it currently knows. But then it may miss other, still unknown options with potential higher reward. Or it could explore the still unknown environment to search for even greater reward. However, it may potentially walk off with even less than when taking the save option.

Solving the dilemma efficiently is difficult, but one intuitive way is as follows: In the beginning, most of the environment, and thus potential reward, is unknown, so the agent has to start by exploring a lot. With time, the agent knows more and more of the environment and can start to utilize its knowledge from time to time. After many iterations, it can then maximize the reward with the known options and only infrequently explore new ones.

The same applies in photography: I could either exploit a known place with known reward, or I could explore an unknown location. If I only choose the first option, I will never find all the beautiful spots out there. And if I always choose the latter, I will miss many good opportunities at good locations while checking out some new places.

Last weekend we decided for the latter: My father and I entered the parking lot at 5 a.m. to, again, get on top of Achtermannshöhe in the Harz Mountains before sunrise (check out Clear Skies and Minus 14 Degrees). From the weather forecast it was unclear if we will be engulfed in clouds or if the clear sky would stretch out above us. Luckily, it was mostly the latter, with some distant orange strips of haze and clouds illuminated by the rising sun. We persevered for 90 minutes in the freezing cold with numb hands and feet, but a breathtaking view made it worth it. In the North, the silhouette of the Brocken towers; in the East, black trees in contrast to orange and purple plains stretching behind; rolling hills dipped into pastel colors in the South; and in the West, the Upper Harz in blue and purple with alternating rows of dead and healthy trees.

After the hike back down we entered the parking lot for a second time, but this time with good memories and full SD cards. For the next adventure, I think I will choose exploration instead.