Object Detection
With over 200 new classes of objects, the Object Detection subsystem enhances Lightship's contextual awareness capabilities by creating semantically labeled 2D bounding boxes that dynamically update as real-world objects appear on-screen. For each bounding box, the subsystem makes an independent prediction for every subclass, returning the probability that the detected object belongs to each of them.
To learn how to access the object detection categories, see How to Enable Object Detection.
Object Detection Categories
While the probability of each class is computed independently, the subclasses of each category are used to train the categorical classes they belong to. Because of this, objects can be detected as members of their categorical classes rather than their specific subclasses. For example, a French horn will return a high probability that the object is in the french horn, brass instrument, and musical instrument classes.
| Category | Subclasses |
|---|---|
| Aircraft | airplane, helicopter, hot air balloon, parachute, rocket |
| Car | car, taxi |
| Vehicle | vehicle, bicycle, bus, car, motorcycle, taxi, train, truck |
| Footwear | footwear, roller skate |
| Headwear | headwear, fedora |
| Musical Instrument | accordion, brass instrument, drum, flute, piano, string instrument |
| Brass Instrument | french horn, saxophone, trombone, trumpet |
| String Instrument | banjo, cello, harp, guitar, violin |
| Food | food, apple, banana, berry, bread, broccoli, cake, carrot, cheese, citrus, coconut, dessert, donut, egg, fast food, grape, hamburger, hot dog, ice cream, pear, pizza, pumpkin, sandwich, sushi, tomato |
| Berry | raspberry, strawberry |
| Citrus | grapefruit, lemon, lime, orange |
| Dessert | dessert, cake, donut, ice cream |
| Fast food | fast food, french fries, hot dog, pizza, hamburger, sandwich |
| Pumpkin | pumpkin, squash |
| Drink | drink, hot drink, juice |
| Hot Drink | tea, coffee (recognized when in a cup) |
| Cooking Pan | frying pan, pressure cooker, slow cooker, waffle iron, wok |
| Furniture | furniture, bed, chair, couch, shelves, storage cabinet, table |
| Jug | jug, teapot |
| Lamp | lamp, candle |
| Screen | computer display, tablet, TV |
| Toy | toy, doll, teddy bear |
| Water Feature | fountain, swimming pool |
| Animal | animal, alpaca, bear, big cat, bird, camel, cat, cow, crocodile, deer, dog, dolphin, elephant, fish, frog, giraffe, goldfish, hippopotamus, horse, jellyfish, kangaroo, panda, parrot, pig, polar bear, rabbit, reptile, rhinoceros, seal, sheep, shellfish, squirrel, turtle, water bird, whale, zebra |
| Alpaca | alpaca, llama |
| Big Cat | cheetah, jaguar, leopard, lion, lynx, tiger |
| Bird | bird, parrot, water bird |
| Crocodile | crocodile, alligator |
| Deer | antelope, deer, moose |
| Flower | flower, rose, sunflower |
| Horse | donkey, horse, mule |
| Insect | insect, butterfly |
| Fish | fish, goldfish, manta ray, seahorse, squid |
| Reptile | reptile, crocodile, turtle |
| Seal | seal, sea lion, walrus |
| Sheep | goat, sheep |
| Shellfish | crab, lobster, oyster, shrimp, snail, starfish |
| Turtle | sea turtle, tortoise |
| Water Bird | duck, goose, swan |