Exploration / Exploitation

Exploration / Exploitation

Reinforcement Learning is one of three main approaches for machine learning and can be described as follows: An autonomous agent observes the environment and performs actions to reach a pre-defined goal. A reward function gives feedback to the agent according to how close it was to reach the desired goal; otherwise, the agent has no information on which actions lead to the largest reward. The agent tries to maximize its gained reward in each learning iteration. And in every learning iteration the agent is caught in the exploration / exploitation dilemma: It could either exploit the already gained knowledge of the environment to safely receive the highest reward that it currently knows. But then it may miss other, still unknown options with potential higher reward. Or it could explore the still unknown environment to search for even greater reward. However, it may potentially walk off with even less than when taking the save option.

Solving the dilemma efficiently is difficult, but one intuitive way is as follows: In the beginning, most of the environment, and thus potential reward, is unknown, so the agent has to start by exploring a lot. With time, the agent knows more and more of the environment and can start to utilize its knowledge from time to time. After many iterations, it can then maximize the reward with the known options and only infrequently explore new ones.

The same applies in photography: I could either exploit a known place with known reward, or I could explore an unknown location. If I only choose the first option, I will never find all the beautiful spots out there. And if I always choose the latter, I will miss many good opportunities at good locations while checking out some new places.

Last weekend we decided for the latter: My father and I entered the parking lot at 5 a.m. to, again, get on top of Achtermannshöhe in the Harz Mountains before sunrise (check out Clear Skies and Minus 14 Degrees). From the weather forecast it was unclear if we will be engulfed in clouds or if the clear sky would stretch out above us. Luckily, it was mostly the latter, with some distant orange strips of haze and clouds illuminated by the rising sun. We persevered for 90 minutes in the freezing cold with numb hands and feet, but a breathtaking view made it worth it. In the North, the silhouette of the Brocken towers; in the East, black trees in contrast to orange and purple plains stretching behind; rolling hills dipped into pastel colors in the South; and in the West, the Upper Harz in blue and purple with alternating rows of dead and healthy trees.

After the hike back down we entered the parking lot for a second time, but this time with good memories and full SD cards. For the next adventure, I think I will choose exploration instead.

Finding Balance

Finding Balance

The composition of a photo can be balanced when the objects are equally positioned in the frame; the colors can be balanced by choosing a consistent or diverging color palette; the brightness can be balanced when the luminosities are equally distributed. Photos that are in balance can look more pleasing or calm and can help to guide the viewer through the frame. However, imbalance of specific aspects can also be used deliberately: to concentrate focus on specific areas, to evoke feelings of discomfort, or to cause tension.

In mathematics, when a target function is non-convex there are multiple local optima and finding the global optimum is difficult. Imagine the landscape around you: If you always ascend in the steepest direction you will likely end up on a hill, but not the highest one. And then it’s hard to leave, because at first, all surrounding areas will be lower. To reach the global optimum, you need to adapt your strategy and find balance: Where should you start your search? How large should be the steps you are taking? Is the steepest direction always the best one? And might it be worth to leave the current local optimum to find a better one?

Balance is important in life: Most of all, balance between work and spare time, but also between friends and family, between body and mind, balanced sleep, balanced food, and so much more. When I am working long hours, I am craving for some free time; when I have longer stretches of free time, I get restless or feel useless. As in photography, the difficult part is to choose what should be in balance right now and in which aspects some imbalance might be fine: How to choose the right colors? How to tune the brightness? How to position all objects of interest in one single frame? And as in mathematics, the path of finding the right balance is uncertain as well: When to leave a local optimum? When to take larger steps? How to find the global optimum? In life even more so, because the landscape is not static with the highest peak fixed at a certain position instead the landscape changes with time and a high peek once found may become a valley over time.

When I have been outside in the recent weeks to find some balance, I have been mainly shooting early spring flowers. But we also visited lambs in the neighborhood and goose are roaming the fields where we often go for an evening walk.

Love is in the Air

Love is in the Air

Last Sunday, I explored a small nature reserve close to our home I haven’t visited before. It’s characterized by small, shallow lakes, surrounded by reed belts and swampy meadows. The weather was rather disappointing but also typical for such an early spring day. The wind was blowing strong and cold from the east and the clouds scurried quickly across the dark sky. I had abandoned the first lens I bought back in October last year in favor of my newer ones, but I retrieved it for that day. It was an enjoyable change, trying to shoot some simple landscapes and to capture the mood of the moment. I also experimented quite a bit with some of the more hidden options of my camera and felt the black and white mode was appropriate for the scenery:

The day before, I had also spotted a lonely stork at the same nature reserve when driving by. He was on a wooden post that had not been occupied the year before, but I stopped only briefly for some quick photos:

So on Sunday I was hoping to meet him again to shoot some additional pictures. I was not only not disappointed, but this time he also had company. And both of them seemed to be neither interested in the dull weather and the light rain nor in the cautious photographer. Instead, they were rather occupied with themselves. They checked their nest under construction, watched each other gently, and synchronized their movements (intentionally, as it seemed to the external observer). In these situations, it is easy to forget the weather, the rain, the place, and time. I hope I can come back soon, and if the weather plays along, I am hoping for a picture of the two in front of the full moon next week.

At the moment I often feel as fortunate as I imagine these two storks feel in their newly built home. I also have someone to share home with, someone to fool around with, someone to synchronize with. It takes time, and sometimes it takes effort, but most of all it doesn’t take, but it gives: It gives moments of bliss and happiness and moments of gratitude and comfort. It’s the short daily breakfast, and the long adventures outdoors; it’s the willingness to try something new, and the teasing and playful banter; it’s the more profound conversation, and it’s the silent walks. And I feel my love growing more and more from day to day.

Fog Clears the View

Fog Clears the View

Forecast: Clear skies, sun the whole day, frosty temperatures in the morning. Reality: Dense fog, no sun for the first hours of daylight, temperatures not cold enough for frost.

That’s how it mostly is – it never comes as expected. I still have to learn to adapt my expectations from forecast to reality. On this particular morning I hoped for ice crystals in front of the rising sun – what I got instead were goose painted on a blank canvas.

Fog drowns the noise and highlights only what is in front of you. And sometimes the unexpected is even better than the forecast: The grey heron was hunting mice directly in front of me and I was able to get closer than ever before.

Know Your Resources

Know Your Resources

On an ordinary day during the last autumn, I saw, for the first time, the common kingfisher – what a beautiful bird. Despite its divergent blue and orange coloring, it is quite hard to spot when sitting still. Only when the kingfisher changes its branch from which it hunts you see a brief blue shimmer darting close above the water. It belongs to the family Alcedinidae whose species are scattered across the whole globe; and most of them are at least as colorful as the common kingfisher (e.g. check out the oriental dwarf kingfisher!). Since then, we have seen the common kingfisher multiple times at a lake close to our home and I have tried to get in on camera at multiple other locations around our town. I spoke with others where to find it, I spent lots of time waiting for it, and I made hundreds of photos of empty branches and little blue dots in the far distance. On some occasions I was somewhat successful, but the clear sight was always interrupted by branches at the locations I visited. Then, last Friday evening after another day of home office, I sought out one of the last spots around our town I haven’t been before during my search for the kingfisher. It’s only 3 minutes from my normal working location but due to the current situation I haven’t been there for a year. And there, directly at a small pond, the perfect location for the king fisher is prepared: A stick curved above the water, a sign that warns uninterested bystanders of the curiosity, and nearby benches and bushes for the interested photographer. I guess, I have to come back next autumn and try my luck here; in the photos below you see my best attempts from this winter.

Work often feels similar to this experience: You search something for a long time before you unexpectedly find it somewhere else. And sometimes you find even more: In this case it was a wonderful sunset and the first spring flowers:

Black and White Spring

Black and White Spring

I don’t like thinking in black and white. I don’t think liking black and white-thinking is sensible: There is always an in-between, an uncertainty, missing knowledge. My side is not correct and your side is not wrong, truth is on both sides. While it’s easy to say, it’s hard to act on: Being empathetic in heated situations, seeing purpose when everything seems empty, or noticing progress when stagnation feels permanent – it takes effort to spot and point at the grey areas. It is exhausting to discuss the subtleties and understand the intricacies of the in-between.
And still, sometimes it is necessary to treat things as if they were black and white, because otherwise we would still be discussing the shades of grey – as long as we are aware of the underlying simplifications. Maybe we should just do it as in modern photography: Take the picture in color, from purple to pink, from dark corners to bright skies, include all shades in between. And then: Edit and redact it, remove undesired colors, increase the contrast, do it careful, and you are left with a delicate black and white version of truth while knowing the steps you took to get there.

I am very happy how the images from today’s morning walk turned out: First, everything looked quite dull and I couldn’t find anything interesting. On my first passing of this field, where small water droplets gathered at the growing seeds, I quickly moved onward when the first pictures did not turn out as expected. I had the wrong settings and not enough patience to find an interesting perspective. Then, on my second passing on my way home, I tried it again and this time I did not want to give up. And after many, many failed attempts of pictures that only showed grass and water droplets, I was able (at least to some extent) to capture the beauty I was hoping to find: Small sceneries of nature in light and shadow that allow the imagination to roam free. Now, I not only see grass and water droplets, but I can see small individuals, I can see groups of fairy-tale characters, I can see them holding lanterns to find there way through illuminated towns in the dark between moon-lit flowers.