This Android keyboard lets you take pictures, labels their contents, and converts the labels to emojis.
How does it work?
When you take a picture, it’s sent to the Google Cloud Vision Api, which returns a list of potential labels. Once the app has a list of labels, it compares them to a local emoji map, or uses emoji-java if the labels aren’t in the local map.
The main difficulty is that Google’s labels rarely correspond to emojis. To fix this, I send a firebase analytics event each time a label isn’t recognized. When I check all of the events later, I just add it to the map. It’s a pretty manual process, but it didn’t take long to add a good base list of emojis.
When I tweeted about 😀⌨️, Frank van Puffelen from Google showed me a fun session from I/O 2017 in which an Android, iOS, and web dev all made a similar service using Firebase. By using Firebase’s cloud functions, they were able to have a single service that performed all of the emojification for all three platforms. If I used a system like that, I could add more emojis without having to deploy a new release of the app.