3. Learning from partial meanings

This learning mechanism creates new constructions that can combine with existing constructions in order to comprehend the observed utterance. The item-based and lexical constructions that were acquired in the previous interactions allow the agent to understand already parts of novel utterances. This allows the agent to retrieve already parts of the meaning of the observed question. When reconstructing a meaning hypothesis, intention reading makes use of these partial meanings in order to rule out large parts of the search space. Concretely, any meaning hypothesis that does not contain the partial meaning can immediately be ruled out. This is how the interplay between intention reading and pattern finding allows to overcome the intractability of the intention reading process. In what follows, partial meaning provided by lexical constructions (3.1) and by an item-based construction (3.2) is demonstrated.

3.1 Partial meaning provided by lexical construction(s)

In the following interaction, partial meaning is provided by lexical constructions. The agent can acquire an item-based construction with an equal number of slots.

Interaction 7

The tutor is the speaker.

The learner is the listener.

Current Scene:

Topic:

shape-category

Question: "What shape is the big cyan object?"


Comprehending "What shape is the big cyan object?"


Applying
g2big-3541what-material-is-the-?x-cube-152->10.300cyan-39991->00.10
in comprehension


Computing next composer solution

using ontology:

and partial meaning:

gstruct0(bindcolor-category?category-376cyan)struct1(bindsize-category?category-377large)

Result

composition process

Composer queue:

... and 2 more

Found 1 solutions:


Evaluating irl program

irl program
((bind attribute-category ?attribute-352 shape) (query ?answer-228 ?source-object-256 ?attribute-352) (unique ?source-object-256 ?source-set-567) (filter ?source-set-567 ?source-set-568 ?category-373) (filter ?source-set-568 ?source-set-548-19 ?category-375) (get-context ?source-set-548-19) (bind color-category ?category-375 cyan) (bind size-category ?category-373 large))
gstruct0(bindattribute-category?attribute-352shape)struct1(query?answer-228?source-object-256?attribute-352)struct0:varATTRIBUTEdash3521->struct1:varATTRIBUTEdash3522struct2(unique?source-object-256?source-set-567)struct1:varSOURCEdashOBJECTdash2562->struct2:varSOURCEdashOBJECTdash2563struct3(filter?source-set-567?source-set-568?category-373)struct2:varSOURCEdashSETdash5673->struct3:varSOURCEdashSETdash5674struct4(filter?source-set-568?source-set-548-19?category-375)struct3:varSOURCEdashSETdash5684->struct4:varSOURCEdashSETdash5685struct7(bindsize-category?category-373large)struct3:varCATEGORYdash3734->struct7:varCATEGORYdash3738struct5(get-context?source-set-548-19)struct4:varSOURCEdashSETdash548dash195->struct5:varSOURCEdashSETdash548dash196struct6(bindcolor-category?category-375cyan)struct4:varCATEGORYdash3755->struct6:varCATEGORYdash3757

Applying



on the ontology


evaluation process

Solutions:

solution 1
?answer-228
shape-category
score: 1.000
?attribute-352
attribute-category
score: 1.000
?category-375
color-category
score: 1.000
?category-373
size-category
score: 1.000
?source-set-548-19
clevr-scene
score: 1.000
?source-set-568
clevr-object-set
score: 1.000
?source-set-567
clevr-object-set
score: 1.000
?source-object-256
clevr-object
score: 1.000


initial structure
application process
applied constructions
resulting structure

Meaning:

gstruct0(bindattribute-category?attribute-352shape)struct1(query?answer-228?source-object-256?attribute-352)struct0:varATTRIBUTEdash3521->struct1:varATTRIBUTEdash3522struct2(unique?source-object-256?source-set-567)struct1:varSOURCEdashOBJECTdash2562->struct2:varSOURCEdashOBJECTdash2563struct3(filter?source-set-567?source-set-568?category-373)struct2:varSOURCEdashSETdash5673->struct3:varSOURCEdashSETdash5674struct4(filter?source-set-568?source-set-548-19?category-375)struct3:varSOURCEdashSETdash5684->struct4:varSOURCEdashSETdash5685struct7(bindsize-category?category-373large)struct3:varCATEGORYdash3734->struct7:varCATEGORYdash3738struct5(get-context?source-set-548-19)struct4:varSOURCEdashSETdash548dash195->struct5:varSOURCEdashSETdash548dash196struct6(bindcolor-category?category-375cyan)struct4:varCATEGORYdash3755->struct6:varCATEGORYdash3757

The following construction(s) were learned:

The following link(s) were added to the categorial network:

g2big-3543what-shape-is-the-?y-?x-object-(?y)-102->30.200cyan-39994what-shape-is-the-?y-?x-object-(?x)-100->40.20

Comprehension succeeded

Interpretation finished

Computed answer:

shape-category

Interaction failed


3.2 Partial meaning provided by item-based construction

In the following interaction, partial meaning is provided by an item-based construction. This learning mechanism can only apply if a single lexical item is missing, otherwise there would be referential uncertainty. In this case, the agent can acquire a single lexical construction. For the purposes of this demonstration, the agent keeps observing the same question until its hypothesis for the lexical construction corresponds to the ground truth.

Interaction 8

The tutor is the speaker.

The learner is the listener.

Current Scene:

Topic:

material-category

Question: "What material is the gray cube?"


Comprehending "What material is the gray cube?"


Applying
g2big-3541what-material-is-the-?x-cube-152->10.300cyan-39991->00.103what-shape-is-the-?y-?x-object-(?y)-103->20.204what-shape-is-the-?y-?x-object-(?x)-104->00.20
in comprehension


Computing next composer solution

using ontology:

and partial meaning:

gstruct0(get-context?source-set-548-2)struct1(filter?source-set-548?source-set-548-2?category-363-2)struct0:varSOURCEdashSETdash548dash21->struct1:varSOURCEdashSETdash548dash22struct2(filter?source-set-547?source-set-548?category-363)struct1:varSOURCEdashSETdash5482->struct2:varSOURCEdashSETdash5483struct6(bindshape-category?category-363-2cube)struct1:varCATEGORYdash363dash22->struct6:varCATEGORYdash363dash27struct3(unique?source-object-247?source-set-547)struct2:varSOURCEdashSETdash5473->struct3:varSOURCEdashSETdash5474struct4(query?answer?source-object-247?attribute-343)struct3:varSOURCEdashOBJECTdash2474->struct4:varSOURCEdashOBJECTdash2475struct5(bindattribute-category?attribute-343material)struct4:varATTRIBUTEdash3435->struct5:varATTRIBUTEdash3436

Result

composition process

Composer queue:

... and 2 more

Found 4 solutions:

... and 1 more

Evaluating irl program

irl program
((bind color-category ?category-374 gray) (get-context ?source-set-548-14) (filter ?source-set-565 ?source-set-548-14 ?category-363-14) (filter ?source-set-566 ?source-set-565 ?category-374) (unique ?source-object-255 ?source-set-566) (query ?answer-227 ?source-object-255 ?attribute-351) (bind attribute-category ?attribute-351 material) (bind shape-category ?category-363-14 cube))
gstruct0(bindcolor-category?category-374gray)struct3(filter?source-set-566?source-set-565?category-374)struct0:varCATEGORYdash3741->struct3:varCATEGORYdash3744struct1(get-context?source-set-548-14)struct2(filter?source-set-565?source-set-548-14?category-363-14)struct1:varSOURCEdashSETdash548dash142->struct2:varSOURCEdashSETdash548dash143struct2:varSOURCEdashSETdash5653->struct3:varSOURCEdashSETdash5654struct7(bindshape-category?category-363-14cube)struct2:varCATEGORYdash363dash143->struct7:varCATEGORYdash363dash148struct4(unique?source-object-255?source-set-566)struct3:varSOURCEdashSETdash5664->struct4:varSOURCEdashSETdash5665struct5(query?answer-227?source-object-255?attribute-351)struct4:varSOURCEdashOBJECTdash2555->struct5:varSOURCEdashOBJECTdash2556struct6(bindattribute-category?attribute-351material)struct5:varATTRIBUTEdash3516->struct6:varATTRIBUTEdash3517

Applying



on the ontology


evaluation process

Solutions:

solution 1
?answer-227
material-category
score: 1.000
?category-374
color-category
score: 1.000
?attribute-351
attribute-category
score: 1.000
?category-363-14
shape-category
score: 1.000
?source-set-548-14
clevr-scene
score: 1.000
?source-set-565
clevr-object-set
score: 1.000
?source-set-566
clevr-object-set
score: 1.000
?source-object-255
clevr-object
score: 1.000


initial structure
application process
applied constructions
resulting structure

Meaning:

gstruct0(bindcolor-category?category-374gray)struct3(filter?source-set-566?source-set-565?category-374)struct0:varCATEGORYdash3741->struct3:varCATEGORYdash3744struct1(get-context?source-set-548-14)struct2(filter?source-set-565?source-set-548-14?category-363-14)struct1:varSOURCEdashSETdash548dash142->struct2:varSOURCEdashSETdash548dash143struct2:varSOURCEdashSETdash5653->struct3:varSOURCEdashSETdash5654struct7(bindshape-category?category-363-14cube)struct2:varCATEGORYdash363dash143->struct7:varCATEGORYdash363dash148struct4(unique?source-object-255?source-set-566)struct3:varSOURCEdashSETdash5664->struct4:varSOURCEdashSETdash5665struct5(query?answer-227?source-object-255?attribute-351)struct4:varSOURCEdashOBJECTdash2555->struct5:varSOURCEdashOBJECTdash2556struct6(bindattribute-category?attribute-351material)struct5:varATTRIBUTEdash3516->struct6:varATTRIBUTEdash3517

The following construction(s) were learned:

The following link(s) were added to the categorial network:

g5gray-31401what-material-is-the-?x-cube-155->10.20

Comprehension succeeded

Interpretation finished

Computed answer:

material-category

Interaction failed


Interaction 9

The tutor is the speaker.

The learner is the listener.

Current Scene:

Topic:

material-category

Question: "What material is the gray cube?"


Comprehending "What material is the gray cube?"


Applying
g2big-3541what-material-is-the-?x-cube-152->10.305gray-31405->10.200cyan-39991->00.103what-shape-is-the-?y-?x-object-(?y)-103->20.204what-shape-is-the-?y-?x-object-(?x)-104->00.20
in comprehension


Evaluating irl program

irl program
((bind color-category ?category-374 gray) (get-context ?source-set-548-14) (filter ?source-set-565 ?source-set-548-14 ?category-363-14) (filter ?source-set-566 ?source-set-565 ?category-374) (unique ?source-object-255 ?source-set-566) (query ?answer-227 ?source-object-255 ?attribute-351) (bind attribute-category ?attribute-351 material) (bind shape-category ?category-363-14 cube))
gstruct0(bindcolor-category?category-374gray)struct3(filter?source-set-566?source-set-565?category-374)struct0:varCATEGORYdash3741->struct3:varCATEGORYdash3744struct1(get-context?source-set-548-14)struct2(filter?source-set-565?source-set-548-14?category-363-14)struct1:varSOURCEdashSETdash548dash142->struct2:varSOURCEdashSETdash548dash143struct2:varSOURCEdashSETdash5653->struct3:varSOURCEdashSETdash5654struct7(bindshape-category?category-363-14cube)struct2:varCATEGORYdash363dash143->struct7:varCATEGORYdash363dash148struct4(unique?source-object-255?source-set-566)struct3:varSOURCEdashSETdash5664->struct4:varSOURCEdashSETdash5665struct5(query?answer-227?source-object-255?attribute-351)struct4:varSOURCEdashOBJECTdash2555->struct5:varSOURCEdashOBJECTdash2556struct6(bindattribute-category?attribute-351material)struct5:varATTRIBUTEdash3516->struct6:varATTRIBUTEdash3517

Applying



on the ontology


evaluation process

Solutions:

solution 1
?answer-227
material-category
score: 1.000
?category-374
color-category
score: 1.000
?attribute-351
attribute-category
score: 1.000
?category-363-14
shape-category
score: 1.000
?source-set-548-14
clevr-scene
score: 1.000
?source-set-565
clevr-object-set
score: 1.000
?source-set-566
clevr-object-set
score: 1.000
?source-object-255
clevr-object
score: 1.000


initial structure
application process
applied constructions
resulting structure

Meaning:

gstruct0(bindcolor-category?category-374gray)struct3(filter?source-set-566?source-set-565?category-374)struct0:varCATEGORYdash3741->struct3:varCATEGORYdash3744struct1(get-context?source-set-548-14)struct2(filter?source-set-565?source-set-548-14?category-363-14)struct1:varSOURCEdashSETdash548dash142->struct2:varSOURCEdashSETdash548dash143struct2:varSOURCEdashSETdash5653->struct3:varSOURCEdashSETdash5654struct7(bindshape-category?category-363-14cube)struct2:varCATEGORYdash363dash143->struct7:varCATEGORYdash363dash148struct4(unique?source-object-255?source-set-566)struct3:varSOURCEdashSETdash5664->struct4:varSOURCEdashSETdash5665struct5(query?answer-227?source-object-255?attribute-351)struct4:varSOURCEdashOBJECTdash2555->struct5:varSOURCEdashOBJECTdash2556struct6(bindattribute-category?attribute-351material)struct5:varATTRIBUTEdash3516->struct6:varATTRIBUTEdash3517

Comprehension succeeded

Interpretation finished

Computed answer:

material-category

Alignment started

The following cxns are rewarded:

Interaction succeeded


Back to top of this page

Back to the main page.