Exploring the Challenges Towards Lifelong Fact Learning

Mohamed Elhoseiny, Francesca Babiloni, Rahaf Aljundi, Marcus Rohrbach, Manohar Paluri, Tinne Tuytelaars

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

So far life-long learning (LLL) has been studied in relatively small-scale and relatively artificial setups. Here, we introduce a new large-scale alternative. What makes the proposed setup more natural and closer to human-like visual systems is threefold: First, we focus on concepts (or facts, as we call them) of varying complexity, ranging from single objects to more complex structures such as objects performing actions, and objects interacting with other objects. Second, as in real-world settings, our setup has a long-tail distribution, an aspect which has mostly been ignored in the LLL context. Third, facts across tasks may share structure (e.g., ⟨ person, riding, wave ⟩ and ⟨ dog, riding, wave ⟩ ). Facts can also be semantically related (e.g., “liger” relates to seen categories like “tiger” and “lion”). Given the large number of possible facts, a LLL setup seems a natural choice. To avoid model size growing over time and to optimally exploit the semantic relations and structure, we combine it with a visual semantic embedding instead of discrete class labels. We adapt existing datasets with the properties mentioned above into new benchmarks, by dividing them semantically or randomly into disjoint tasks. This leads to two large-scale benchmarks with 906,232 images and 165,150 unique facts, on which we evaluate and analyze state-of-the-art LLL methods.
Original languageEnglish (US)
Title of host publicationLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
PublisherSpringer Verlagservice@springer.de
ISBN (Print)9783030208752
DOIs
StatePublished - Jan 1 2019
Externally publishedYes

Fingerprint

Dive into the research topics of 'Exploring the Challenges Towards Lifelong Fact Learning'. Together they form a unique fingerprint.

Cite this