The cutting edge of race photo tagging software
We've been working hard this winter to release cutting edge software that will bring race photography into the twenty-first century. While many underestimate what it takes to get tagged photos into your email inbox, hours of work are involved between the time you lace up your shoes and to the minute you get the post that great photo on Facebook. However, we wanted to detail the improvements we've made on the tech side, as well as how we're helping photographers, race directors, and timers handle photography, so that next time they need a race photo tagging solution, they know where to look.
I'm sure many have seen, we released a beta version of shirt color detection, which we will be able to confidently say is in full production within two months, Nate went into detail on this in a previous blog post, but we've gotten questions on how the feature came to be built. We use deep learning, also known as machine learning; the algorithm is trained to identify faces and run that over the image. Once we can accurately find a face, we tag the closest color to a predetermined palette and let you select! In simple, we detect the face and find your shirt color.
We've been working hard and taking our knowledge of machine learning and applying that to create the best bib detection algorithm that currently exists in the world. We've trained our algorithms to be able to detect a bib out of any picture, regardless of color, shape, size, font, et cetera, then once we can effectively do that, we go ahead and read it. This is a different and more successful approach than what's traditionally being done by other companies because its specifically made around detecting bibs. There are off the shelf optical character recognition programs like Google and Abbyy, we built one ourselves when we started, but this approach requires huge amounts of code, large amount of image manipulations, and at the end of the day was pretty slow. The off the shelf solutions are tailored for reading flat pieces of paper, not runners. Now granted, we aren't completely abandoning that approach, image processing is a key part of the project, and we are using that knowledge in creating a better approach with machine learning and OCR to create one heck of a working solution. Stay on the lookout as we tailor our solution more specifically for sports that do not rely as heavily on bibs for identification such as triathlons, biking, and skiing.
Currently our biggest competition is human manpower, as that's currently the most accurate method to tag, but definitely not the most scalable solution. While human labor is accurate, photos can take days to be tagged by people, and we've seen at the longest up to a week. In the worst situations, we've seen this process take a week, and in some cases the tagging is done hastily resulting in many untagged photos; we've seen this is the case when using 'software' solutions like Amazon's Mechanical Turk. This ultimately delays sale of photos, hurting both the photographer and race in potential profit. The desire for photos is a fleeting thing, we've seen photo sales peak within 2-3 days after the event occurs; so make sure you get your photos tagged and ready to go as early as possible. Finally, the cost should be the most concerning to photographers and race directors. Human tagging is expensive. While flat rates can be negotiated by some companies, we've also seen companies charge per bib tag, at between 1 to 3 US cents per tag. Though at first glance, it sounds reasonable, most photos have two or three athletes per photo, meaning at best you are paying around 20 dollars per hour for around 700 photos for runners to be identified. Ultimately this eats into margins, and means less for the event.
With an automated solution we can accurately tag thousands of photos within seconds of upload, and have them ready to be purchased immediately. When used within the Flashframe platform, we don't charge extra for the tagging service. Don't have human tagging be the blocker in sales pipeline, with Flashframe's automated bib tagging software you can be selling photos as soon as you upload, maybe right after a little bit of photo editing? We shift the blocker from the tagging, to just the uploading, so make sure you got a fast internet connection! Additionally, we have the added bonus of cool additions like the earlier discussed shirt color detection.
We've been asked by a few photographers and race timing companies if we allow use of our software outside of the Flashframe platform, we're proud and excited to say in the coming months, yes. Now instead of using Chronotrack live, or Abbyy to tag your photos, by time or by bib, we'll soon allow full access to the Flashframe API. Just send us your photo, we'll tag, and send you back the readable bib tags. For those photographers that are interested, we're also planning to release a version that will write the relevant bib numbers to the photo's exchangeable image file format, or exif data. Don't expect magic though, we can't read bibs that are blurry, and keep in mind, if you can't read it, it won't tag. Finally, unlike you and I, our software API doesn't have memory between photos, so if a bib can only be inferred between two photos, don't expect it to happen. Maybe consider using the Flashframe site for cool stuff like that...
So how far off is face recognition? That is pretty difficult to do, and our situation (people grimacing while running) makes it all the more difficult. The key to face recognition being successful in a race photography would the participants consistently using the platform. This requires building a database of photos of a person, and then being able to store the salient features of them (while they are running), then trying to do the matching. The most successful company doing matching like this is Facebook or Apple, and even when they make educated guesses on the hundreds of photos they have of you, its sometimes wrong. Think about our possible situation if we were to put this into practice: we have at most a dozen photos of a person, ask them to upload a photo of themselves (probably with a different facial expression than them running), then find the match. Needless to say, it would be incredibly difficult, but definitely not impossible. We'll continue to focus on the most easily accessible distinguishing features for now, and make sure we can execute at the highest level before we seek new research and development adventures.
We're entering a new age of software as machine learning becomes the norm within technology. As other consumer industries advance themselves and machine learning is slowly introduced, endurance sports like track and field, running, biking, and triathlons, as well as the individuals that host these events need to understand and embrace the technological advantages that are being put forth. We anticipate Flashframe to be on the forefront of image recognition within race photography, and soon, more generally event photography, so look forward to the advancements we put forth in the new future.