eBay’s New Image Search – A Hands-On Test of The Feature

At eBay Open 2017, the company showed off image search and promised to make it available later in the year. They followed through on that pledge, and it is now part of the eBay app.

So I decided it would be fascinating to run a set of tests to see how well image search performs.

The Test Methodology

This is no scientific test by any means, but I thought I would try to use my phone in average lighting conditions with imperfect backgrounds. I even forced some image warping as my phone camera will go into a macro mode for up-close images.

Most users are going just to snap pictures and go, so that was the idea I wanted to try with my experiment.

I also did not crop any images or alter them in any way for the search results. For each test below, I provided the actual image used for the search.

Women’s Boot Style Shoe

The first test object for the new eBay image search was a fashion item. I think we can all agree that shoes probably are going to be high on the list when it comes to image search.

I took a black boot style shoe from American Rag Cie and searched eBay for it. This shoe was just purchased a few days ago from Macy’s, so it is a current model.

You can see from the three search results images, for the most part, eBay provided me with options that were pretty close.

I also added two images from actual listings, so you could see how close it came

I think it is fair to say the eBay search did pretty well. It did not recognize the actual shoe, but I tried a standard text search and was not able to find any also.

But let’s be fair here, this is an all black shoe, with little detail shown. I give eBay high marks on finding me some similar shoes, and I think for the intended audience the search worked pretty well.

Men’s Shirt Test

In this test, I used a men’s grey Sean John short sleeve shirt with buttons and a collar. The shirt style about a 4 to 5-year-old style, but does this type of shirt really go out of style?

I got results that included men’s and women’s shirts and even some Khaki colored shirts.

I was a bit amused by the High Reflective safety shirt suggestion, but overall the search did pretty well. Is there really a way to differentiate on a shirt like this between men’s and women’s shirts? I doubt it.

All results were short sleeve shirts with buttons and a collar and most of them were gray. While Khaki is not gray, I can see how it may make search results, but more on that later.

No Sean John shirts were listed as a top result, and a text search did find similar looking Sean John shirts. Again, very little detail to use to identify an actual brand, so I think eBay did pretty good serving up results based on the image provided.

Red Jacket Test

A red Tommy Hilfiger jacket from the early 2000s served as my test. I live in Florida, and we don’t use these much, so they last a while 🙂

Again, I got results for unisex, men’s and women’s styles. eBay’s search did pick up on the fact that my search subject had dark accents. So it seems it was trying to give me options with similar accents.

The jacket I used was a light jacket, but with the winter season coming up, the site is full of heavier coats and jackets. You wouldn’t want to go skiing with the jacket I used for the test.

No Tommy Hilfiger results were in the top results, and a text search for red Tommy Hilfiger jackets did get me close duplicates being sold as “Vintage.”

But like the previous two tests, basically, I got options that were similar to the search image.

Kölsch Beer Glass

This test was interesting because I used a tall, thin beer glass (200ml), typically found in the Cologne, Germany region. Also, the beer brand is available in the U.S. at specialty retailers, so only beer fans would know it.

The results surprised me. eBay image search recognized the glass as a beer glass and gave me listings of other beer brand glasses and pitchers. So did eBay recognize the logo as a beer brand logo or did they get lucky?

To answer that question, I took another picture of the same glass but turned in a way that the logo was not visible.

eBay’s search results were now very different. Mostly it realized I was looking for a tall glass, but it even gave me a couple options of with logos on them (NFL New England Patriots and UM – University of Miami).

Did they serve up a UM glass because it knew I was in South Florida? But then why show an NFL New England Patriots glass, especially since they are sworn enemies by many Miami Dolphins fans?

 

This test showed me that in the first beer glass picture with the brand shown, eBay did recognize it as a beer logo.

In the second picture, without the logo, it gave me results based on the shape of the glass. The search even showed a Pilsner beer glass and then provided a few surprises with glasses that had sports team logos.

To me, this test showed a lot is going on with image search. eBay’s search is going beyond basic shape and may include regionalized options. While Dolphins fans may hate the Patriots, there are a good number of New England Patriots fans that live here. Was that considered or just a fluke?

From this test alone, it does appear logo recognition is part of the search technology in some form.

Let’s Test this Logo Theory Again

This time I used a J.A. Henckels Zwilling bread knife and I tried two different images. In the first gallery, the knife’s logo is challenging to identify. In the second gallery, there is more detail visible from the logo and the knife.

You can see in the first gallery, I mostly got a variety of thin blade knives, including flip knives. But no top result was a bread knife or a knife from J.A. Henckels Zwilling.

Here is the second gallery with the logo more visible. The results were better as I got bread knife options and three of them from J.A. Henckels Zwilling.

Two tests now that used images with recognizable logos and it does appear eBay attempts to identify brands from logos.

The Coffee Maker

For the next test, I stuck with kitchen equipment. I took a picture of my often used Keurig coffee maker on my kitchen counter with all the clutter around it.

 

A perfect match on the top search results. eBay provided me with different versions (SKUs) of the same design by Keurig, but that was to be expected as most people wouldn’t be able to know the difference from a photo anyway.

The Keurig logo probably helped too, but good job here finding a replacement if I ever need to change our my coffee maker.

The All-Purpose Pan

Another A/B test. In the first image, I kept the clutter from the original purchase in the pan. Just got this recently as a gift, so I had not used the pan and still have all the paperwork and inserts just as they came out of the box.

In the second test, I removed the clutter and just took a picture of the pan.

 

Here we have the first real failure to identify the image. The first search with the clutter in the pan (image 1) resulted in eBay showing toy boats. Hmmm….?

But once I cleaned up the image (image 3), I got reasonable similar options, but not anything like the pan I used. A simple “square pan with lid” text search resulted in way closer choices.

On this test, I would say I did not get anything like my original subject. I forgive eBay for showing toy boats on the pan with all the clutter, that was pushing recognition.

But I would have thought displaying square style pans as top results should have been part of the top results of my second image used. The closest eBay served up was a rectangular pan.

The $20 Bill

Now, this was another fun one. There are a lot of people selling collectible currency on eBay. So I thought it might be interesting to find out what eBay would serve up when taking a picture of a regular U.S. $20 bill.

 

The top result was an exact match, a regular $20 bill being sold for $26 with free shipping 🙂 The rest of the results were all over the board presenting various forms of old U.S. currency.

So for total accuracy, eBay found an exact match! Would you pay $26 for a $20 U.S. bill? That is one way to turn your PayPal balance into cash. No offer was made for international shipping.

But that odd listing is not eBay’s fault, and if anything, the search did exceptionally well. I got an exact match of the item I was searching, same brand and condition 🙂

The Gallery of Failures

I already pointed out one failure earlier, but eBay still did provide some relevant search results in the pan test (except for the picture with the clutter in the pan, but I already forgave them for it).

I used three objects, a wallet, a battery pack for charging phones and other USB devices, and a Lowe’s brand (Kobalt) standard 25ft tape measure.

eBay identified the wallet as make up bags, the battery pack as TV receivers, and the tape measure as watches. To be fair, I tried a different image on the battery pack that would show the logo and eBay then served up battery packs.

On the wallet, the lack of features may be an issue, so the search technology recognized it as a bag. I am sure if I mucked around with taking different pictures (open wallet); eventually, I would get wallets.

The battery pack looks like a brick. There is not much shape to it, but it does appear the image search recognized “ports.” How did it get to TV receivers?

I think there are lots of other brick style tech devices, from cable modems to routers to external hard drives it could have chosen. I did do a second test (not shown) with a picture that showed the logo much better and I then received battery packs as top search results.

But the tape measure? Full logo and brand were shown, one of the more precise pictures I used for testing and eBay’s search returned watches? That one was a big fail.

More about the failures in my conclusion.

A “Perfect” Image Test

As I mentioned earlier, fashion item searches probably have the most significant use of this feature. I could have gone on to test many other fashion items, especially accessories.

But I wanted to see what would happen if I used perfect images from another website to use on eBay’s image search.

I went over to Amazon on my phone and downloaded images from 2 dresses and 2 skirts for this test. I kept all results in one gallery, so do scroll through it.

As you can see, results were a bit mixed. On the red skirt which included the model, the eBay image search top results were skirts in reddish colors. But the design choices were all over the place.

For the black skirt, the image search did much better. All search results were black skirts. This image was only of a skirt, so the did the model in the first skirt image “confuse” the algorithm?

I tried two more images using models, this time wearing dresses. On the first picture, I used “summer dress” with a floral pattern.

While the search served up summer style dresses, the colors and designs were all over the board again. Worse than on the red skirt!

So for the final “perfect image” test, I tried another even more difficult dress image with the model showing more skin, basically a sexy beach outfit.

The surprising result here, the image search did very well. It pretty much identified the same type of “sexy beach” dress. Even the floral patterns were a much better match, but then again, this kind of outfit commonly use floral patterns.

My twisted thought about why the last image did so much better had me thinking back to the dark ages of programming mostly dominated by male programmers working from mom’s basement…. But I digress

One Final Test – Searching For a Car

I was wondering if eBay Motors would be included in the image searches. Often eBay’s primary platform and eBay Motors are treated a bit differently. Would that apply here, or did eBay roll this out to Motors as well?

So let’s find out.

I started out with the iconic Porsche 911 (991 Series), using one of IMSA’s pace cars. No problem for eBay, lots of Porsche 911’s for sale on the site and the search correctly identified the car model.

It did serve up various series of 911s, but trying to identify the nuances in the design of each series to serve up more of the same is probably asking too much right now.

Let’s try another “iconic” car. Remember the Yugo? It is actually a Zastava Koral but renamed to Yugo for export during its reign here in the U.S.

Anyway, that didn’t work out so well. eBay identified the Yugo as a Chevy Blazer! But to be fair here, no Yugos are for sale on eBay right now.

I do think eBay image search could have served up another small car from that era. Many econoboxes still being sold from VW and Toyota that could be close to the Yugo “style.”

The Final Word

This article ended up longer than I expected, but I tried to give it some depth and insight into the new image search function by eBay.

Kudos for eBay to go out on a limb with this one. Unlike a specialty retailer that can narrow the search to specific items it sells, eBay has to deal with over a billion items, many are amateur images (smartphone pictures), and spread over thousands of different categories.

One could make the point it would have been a lot easier for eBay to roll this out on select categories and tweak the technology for those categories.

Instead, they made it available for all searches, and overall the results are pretty good. I doubt they expected perfection and neither did I when doing this test.

For a very brief comparison, I used image search on Amazon’s app, not widely known it exists, and the results were horrible.

The same black skirt image that I downloaded from their site was identified as toiletry bags. Remember that picture was one of the best results in the eBay image search test!

There is a lot of room for improvement, but we are dealing with a brand new feature pushing the consumer experience in a whole new way.

The search results will improve as eBay gains more images to use for their algorithm. We have seen tremendous leaps in image recognition over the last few years, and with companies pushing visual experiences to new levels, it will improve the breed.

The most impressive part of eBay’s image search for was how well it identified items on imperfect images. The app gives you the ability to crop the object and make it stand out more, but I didn’t use that function at all.

Yes, there were failures in recognizing the actual object. And except for the tape measure, most of them can be explained due to lack of detail in the original image that would identify the item better.

But eBay’s image search does seem to look for logos, either for hints about what the image contains or to serve up items that from that brand.

A user can try to take more images, better close-ups, crop the image, or otherwise adjust the picture and probably get better results. But all that is friction to using the feature in the first place.

For image search to be useful, it needs just to work!

In software development (and other industries), there is a concept of Minimum Viable Product (MVP). This means a company will release a product to the public knowing it has flaws or missing features, but the product is good enough for many users.

eBay met MVP with its image search. I think it is well beyond MVP and is very useful for many categories.

As I said, I still expect a lot of improvement in image search, and that will lead to better search results.

I used an LG V20 smartphone for my test, a relatively capable model to create high-resolution images. There are a lot of people with low-end smartphones that simply cannot provide highly detailed images for search.

For the time being, image search seems to focus on identifying objects with a few hints about brands. Likely this level of recognition works on most phones in use today, and that was probably the goal here.

I give this new feature a big Thumbs Up!

Recognizing it is early days here, but getting this out now before the holidays was brilliant timing. It is a significant feather in eBay’s cap.

From a seller’s point of view, you need to make sure you use items specifics.

There may be some other background operation going on to bring up matches, but I believe eBay’s image search fundamentally tries to recognize essential features of the search object and then serve up a list of items based on items specifics data in the listings.

I also believe some of the failures to show depth in search results is that sellers are not using enough data to help the feature. It is very possible some listings with minimal item specifics information are not shown at all.

I deduce my suspicion by comparing text searches, which focus on the title, and compare the results to image searches. The search results are very different.

What do you think of this new feature? Do you think you can take advantage of it for your products? Let us know in the comments section below.

If you liked this article and would like to engage with other small business entrepreneurs selling on marketplaces, join our . You can also find us on , , , and or sign up for our newsletter below.