Yixin Zhang of our research group gave presentation at ACM Multimedia Asia 2024.
This presentation focused on ingredient recognition from images in recipe data that describe cooking procedures.
Recipe data consists of a sequence of descriptions for each cooking step. As a result, the image for each step often contains ingredients that also appeared in previous steps. However, because recipes can include branching and merging steps, ingredients do not always carry over from the immediately preceding step. To address this, we developed a method that estimates which preceding steps the current step continues from and utilize image information from that step.
Additionally, the ease of recognizing ingredients in cooking images varies significantly depending on the ingredient. Moreover, it varies significantly even for the same ingredient depending on the dish because the appearance of the ingredient may remain unchanged in some dishes, while it may look entirely different in others. This means that recognition thresholds need to be adjusted not only for each ingredient but also for each dish.
To tackle this, we leveraged the fact that in almost all recipe data, the final step image contains all the ingredients used in that dish. Using this insight, we developed a method to determine appropriate recognition thresholds.
The conference was held in Auckland, New Zealand.
It’s December, but since we are in the Southern Hemisphere, it’s a summer Christmas.
In Auckland, there are many volcanic craters within walking distance from the city center.
This photo shows the view from the opposite side. The slope is quite steep. Those visible in the distance are people.
