Archive for the ‘intersubjectivity’ Category


interrelated affordance dimensions as systems

November 4, 2010

I am preparing the virtual lecture “Ecology of learning with new media tools” for the master of semiotics program in Helsinki University for the course “Semiotics and media, sciences and technology studies”.

I was looking one article that was inspired by the Lakoff’s book “Metaphors we live bye”.
It assumes that we live by metaphors that actually structure our perceptions and understanding

Our conceptual system, thus, plays a central role in defining our everyday realities. If we are right in suggesting that our conceptual system is largely metaphorical, then the way we thinks what we experience, and what we do every day is very much a matter of metaphor.

Interesting in this paper is the assumption that metaphorical concepts that we use form a system.

TIME IS MONEY, TIME IS A LIMITED RESOURCE, and TIME IS A VALUABLE COMMODITY are all metaphorical concepts. They are metaphorical since we are using our everyday experiences with money, limited resources, and valuable commodities to conceptualize time. This isn’t a necessary way for human beings to conceptualize time; it is tied to our culture. There are cultures where time is none of these things.

I started to think if there exists also the personal system within the affordances that we potentially actualize in interaction with the world.

My idea seems not to be exactly the same as affordance network idea conceived by Barab and Roth (2006). Particularly it is elaborating this part where environmental knowledge is used.

Barab and Roth (2006) have noted that connecting learners to ecological networks, where they can learn through engaged participation, activates the affordance networks.
Barab and Roth (2006) assumed that affordance networks are not read onto the world, but instead continually “transact” (are coupled) with the world as part of a perception-action cycle in which each new action potentially expands or contracts one’s affordance network.

Basically i think that:
a) if affordances are our perspectives, the positions that we take in the moment of action/emotion in the multidimensional geo-conceptual hybrid space consisting of conceptual dimensions and geographic dimensions (Pata, 2010; Normak, Pata, Kaipainen, forthcoming), then

b) there exists the personal spatial area within geo-conceptual hybrid space that is frequently defined by these positions
This personal spatial area (a cognitive niche) is simultaneously activated internally and externally as the cognitive distributed space during the cognitive chance-seeking (Bardone, 2010), and people are always “validating” the effectiveness of this space for affording their actions and emotions.

c) and within this personal space WE CAN FIND CONSISTENCY of what dimensions of the space are incorporated into certain affordances as personal perspectives useful for certain action or emotion

d) The accumulation of individual positions within this space (to the geographical and virtual object world and to the interpersonal relational actions) contribute to the formation of the cultural spaces – the niches within geo-conceptual hybrid space.
So some of the affordances are offloaded to the objects which are spatially located, some affordances are run dynamically in the awareness of the persons who are interacting and keeping awareness of bodily and emotional activations of each other and with the object world.

We may have several of such taskspaces.
Taskspace is an array of activities related to a certain environment (Ingold, 2000). A taskspace fosters a range of affordances of an environment, delimiting some and enabling others (Edensor, 2004).

e) Cultural niches within geo-conceptual hybrid space are used by individuals for spatial navigation while they select the positions in their own spaces (basically cultural niches can prompt or inhibit some dimensions that the person can use in the geo-conceptual hybrid space for actualizing affordances.

(dataset and image from Pata, 2009)

Image indicates the community perception of affordances for using an aggregator tool.

Part of the problem is how effectiveness of taking action or having emotion is evaluated by each individual in respect to the community niche, and how such effectiveness may be accumulated to the niche.

If the (geo)tags used for defining some conceptual artifacts are interpreted as the dimensions of the geo-conceptual space (for example if we look blog posts, or bookmarks), there exist some dimensions that are the root- or central dimensions, and other dimensions are additional dimensions.

The pictures of tag-networks allow us to see the “hubs” (root-dimensions) in this multidimensional space.

Here is the affordance dimension network based on my dataset (Pata, 2009a,b). I have used the Bayesian networking tool for finding the best fitting causal model for collaborative activity taskspace with social software tools.

From the previous spatial dimension figure we can see that monitoring is the most frequently perceived affordance of the aggregator. The other affordances frequently perceived while using aggregator are: filtering and mashing; collecting; reading; and evaluating.

We may assume that in the collaborative activity taskspace with different types of social software tools, the monitoring affordance in general is related with searching and evaluating and reading.
The arrow to reading indicates causality that actualizing monitoring affordance allows in turn reading affordance.

Following the same idea of spatial re-location while taking action and having emotion, Lackoff said about conceptual metaphors that Another functionality for metaphors is orientation in space.

I’m feeling up. That boosted my spirits. My spirits rose. you’re in high spirits. Thinking about her always gives me a lift. I’m feeling down. I’m depressed. He’s really low these days. I fell into a depression. My spirits sank.

Lakoff and Núñez suggest that conceptual metaphors form network of bodily grounded entities with inferential organization.

In his book “Philosophy in the Flesh : The Embodied Mind and Its Challenge to Western Thought” Lakoff and Johnson (1999) conceptualized living by metaphors using the embodied mind idea.

“our bodies, brains, and interactions with our environment provide the mostly unconscious basis for our everyday metaphysics, that is, our sense of what is real.”

Together with the “father” of embodied simulation Vittorio Gallese George Lakoff wrote and article “The Brain’s Concepts: The Role of the Sensory-Motor System in Conceptual Knowledge.” (2005).

The argue against the cognitive processing

A common philosophical position is that all concepts—even concepts about action and perception—are symbolic and abstract, and therefore must be implemented outside the brain’s sensory-motor system.

and suggest embodied simulation, assuming that

“sensory-motor regions of brain are directly exploited to characterise the so-called “abstract” concepts that constitute the meanings of grammatical constructions and general inference patterns.”

In the recent book “Embodied cognition” Shapiro distinguishes three important themes in embodied cognition (Shapiro, 2010):

Conceptualization – the properties of the organism’s body constrain which concepts an organism can acquire.

Replacement – the organism’s body in interaction with the environment replaces the need for symbolic representational processes. (systems do not include representational states)

Constitution – the body or world plays a constitutive role rather than causal role in cognitive processing.

I am thinking of two interesting aspects:
How is personal cognitive niche/a cultural niche a coherent referential network?

A person can offload some of the affordances to the environment using some artifacts, so the community niche may form and be reused for personal cognitive navigation?

A person interacts with other people directly and the monitored actions and emotions actualize temporarily parts of the community niche as well, which may be used for navigating in personal cognitive niche

How are some dimensions in the geo-cognitive space highlighted among others, and which are in principle these “spaces of flows” within cultural/community spaces and how one person is immersed to these flows.


Socio-cultural and ecological explanations to self-reflection

February 10, 2008

I was reading this sunday morning the chapter from the Cambridge Handbook of Sociocultural Psychology (2007) by (eds.) Jaan Valsiner and Alberto Rosa:

Social basis of self-reflection
by Alex Gillespie


Since i have been thinking in terms of inter-subjectivity, activity theory and cultural semiotics earlier, while now my understanding has more and more shifted towards the embodied cognition and hybrid ecology ideas, i tried to see where my standing-point is and where it differs from socio-cultural ideas.

It seems to me that the basic idea in this chapter is recognizing that signs (but then also tools, since both are mediators of action what person needs to realize his objectives in an environment?) are created during culturally constrained actions as multi-perspective and inter-subjective representations, including both the actor’s and the observer’s experiences of that action.

Gillespie suggests that in different social acts we will get experiences of the both sides of the act in lifetime (learner/teacher, giving/receiving), so we can activate these perspectives simultaneously when the we need to create/activate a mediator (sign, tool) to carry out any act.

The re-using of the signs means activating these embodied experiences and switching between these multiple perspectives when using certain sign either alone or with the others in interaction.

In Gillespie’s elaboration i can see direct relations with embodied cognition and mirror-matching theories: these theories assume that we need to experience something, embody it, and only then we can observe others doing it so that it might reactivate our similar neural processes. But embodied cognition has not dealt with this constant activation of different experiences simultaneously – my own perspective as an actor, and the other’s perspective as an observer of that action.

Secondly, in embodied cognition the representational mediation, the processing of signs that represent something is excluded, and the observation, hearing or reading can directly activate sensory-motor paths that make as feel and act.

Following Gillespie, and relating it how i understand these issues, in case of conscious self-reflective activities we might simultaneously activate several previously embodied affordances of the environment (extracted dimensionalities) to do something what we wish to do (eg. my experience of learning and also my experience of teaching), then we are running these sensory-motor activations in parallel/simultaneously/one-by-one that means as a result that we sometimes suppress some affordances in the environment that we initially perceived as coupling with our anticipated affordances for doing some actions.

Rupture and the use of internalized actions as part of self-reflection in this case are the constraints we put to the anticipated affordances of actions internally before even trying to carry them out. Can it be like conscious hindering certain sensory-motor neural activation patterns as part of our decision-making of what act to perform?

Mirroring from others and the social conflict are the constraints emerging from the environment as the response to find/make use our anticipated affordances of action. It means we consciously accommodate our sensory-motor activation paths ecologically, searching in other people, in the environment for coupling affordances of our anticipated affordances for action and hindering those sensory-motor activation paths that do not find the match to become activated.

These are some ideas what i got reading the following parts from the Gillespie’s article:

Self-reflection can be defined as temporary phenomenological experience in which self becomes an object to oneself.

People use semiotic mediators, or signs by which they pick out certain affective experiences or situations, thus distancing themselves from both self and immediate situation. These signs are combined into complex semiotic systems (representations, discourses, cultural artifacts, symbolic resources), that provide even greater liberation from the immediate situation.

Such distance enables self to act upon self and the situation.

Four socio-cultural theories of the origin of self-reflection:

1. Rupture theories of self-reflection posit that self-reflection arises when one’s path of action becomes blocked or when one faces a decision of some sort.

Peirce: A problematic situation. a small irritation or rupture stimulates reflective thought (1978/1998).

Dewey (1896): in ruptured situations the object becomes subjective because the actor has two or more responses toward the object, and the self-reflection arises.
However, from Pavlov’s experiments it is shown that contradictory responses can co-exist without leading to self-reflection.

According to Piaget (1970) the problem situation forces the child to abstract and recognize his/her developing schemas when these schemas lead to unfulfilled expectations.

It was not clear from this explanation, why semiotic mediators must be stimulated.

2. Mirror theories of self-reflection suggest that the defining feature in self-reflection is the presence of an other.

The other perceives more about self-reflection than self can perceive.
The reflective distance from self which self-reflection entails first exist in the mind of other. This can be fed back to self by other, such that self can learn self from the perspective of other (Bakhtin 1923/1990).
Other provides feedback to the self same as mirror provides feedback about our appearance that we cannot perceive unaided.

The society can be a mirror as well, leading to self-reflection (Cooley, 1902). According to him, self is a social product formed out of our appearance to the other person, the imagination of his judgement of that appearance, and some sort of self-feeling such as pride or mortification.
Cooly always related self-reflection with judgements leading to emotions such as pride, shame, guilt etc.

Questions: How does self take the perspective of the other? Is other a passive mirror, neutrally reflecting back to self?

3. Conflict theories of self-reflection suggest that self-reflection arises through social struggle.

Hegel: self-consciousness arises through gaining recognition from an other who is not inferior to self. Self and other treat each other as physical objects, and thus deny any recognition to each other. Due to this denial they enter into a struggle, the outcome of which is the relation of domination and subordination, that is master-slave relation. The slave can get recognition from the master but not vice versa. Slave struggles for recognition, developing new skills and competences. Self-onsciousness arises from struggling for recognition.

Psaltis & Duveen: Explicit recognition of new acquired knowledge by other and self is needed for durable cognitive development through interaction – the interaction needs to provide mutual self-reflection.

Sigel’s (2002) Psychological Distancing Theory asserts that discrepancies introduced by utterances of others can put a cognitive demand on the child which can in turn lead to representational work and thus distancing.

Activity Theory (Engeström, 1987) assumes that problematic situation includes problems introduced by the perspective of others. Participants within an activity system prompt each other to reflect upon the conditions and rules of their ongoing interaction. Thus contradictions between different counterparts of an activity system lead to reflection.

Social representation theory (Duveen) emphasizes that there are contradictions in the bodies of knowledge that is circulated in modern societies. Bauer and Gaskell (1999) suggest that people become of aware of the representations at the points at which they overlap or contradict each other. This coexistence of multiple forms of knowledge in the society can lead to self-reflection.

Similarly to rupture theories, it is not clear through which semiotic processes self-reflection arises.

4. Internalization theories of self-reflection posit that thought is a self-reflective internal dialogue with absent others, between their internalized perspectives.

Self-reflection arises through internalizing the perspectives that the other has upon self, followed by self taking the perspectives of other upon self.

Vygotsky (1997) emphasized that the process of internalization is a process of transformation rather than simple transmission. Signs are first used to mediate the behaviours of others, and later used to talk about self, reflect upon self, and mediate the behaviour of self.

Mead and Vygotsky conceive the sign (or significant symbol) as comprising two perspectives – the actor perspective and the observer perspective.

On one hand, there is the embodied actor perspective (the response) to some object (the child reaches hand to point to an object she wants to get). On the other hand, there is the distance introduced by the observer perspective of the other on the action (mother sees the grasping gesture indicating desire to get the object). The grasping becomes pointing when the child uses both of these perspectives.

Thus the sign (significant symbol) is fundamentally inter-subjective: it evolves both actor and observer perspectives in both self and other.

Questions: if the sign is composite of the perspective of self and other, how does this composite form, how are these two perspectives brought together.

Gillespie (2005) now starts to generate his own theory. He relies on the Mead’s theory of the social act suggesting that people move amongst the positions with a relatively stable social/institutional structure (host/guest, buyer/seller).

Each social act pairs (eg. giving/receiving, teaching/learning) entails reciprocal actor and observer positions and perspectives which mots people have enacted. They have previously been in these social positions of the other. Thus we are able to take these perspectives in each social act. The self becomes dialogical, containing multiple social perspectives for each act.

The social act is the institution that first provides individuals with roughly equivalent actor and observer experiences, and second, integrates these perspectives within the minds of individuals. When both actor and observer perspectives are evoked within a significant symbol (or sign) /like in gesture/, then there is a self-reflection, because self is both self and other simultaneously.

Gillespie calls self-reflection triggered by an actor perspective self-mediation and the self-reflection triggered by an observer perspective on an actor short-circuiting.

Gillespie assumes that different socio-cultural theories of self-reflection are not in opposition, but rather theorize different proximal paths leading towards self-reflection.

The magic of social act is that it integrates the actor and the observer experiences or perspectives into the formation of signs enabling higher level of semiotic mediation. Conceiving of the sign as this integration of perspectives elucidates the logic of self-reflection.

Whenever one uses the sign it can carry self from one perspective to another continuously.
Introducing the concept of sign (significant symbol) as a complex semiotic system entails abandoning the assumption that complex semiotic systems mirror the world. Instead, it conceptualizes these semiotic systems as architectures of inter-subjectivity, which enable translations between actor and observer perspectives within a social act.

Any narrative is not just a narrative that is analogical to self’s own experience, it is an inter-subjective structure that enables translations between actor and observer perspectives. Partially integrated actor and observer perspectives are the pre-condition for self-reflection. Rupture, feedback, and social conflict can cause self-reflection because of a pre-ecxisting and only partially integrated architecture of inter-subjectivity.


distributed self

November 5, 2007

One of the phenomena in web 2.0 is keeping distributed self.

We all invade various spaces: weblogs, twitter, jaiku, flickr, youtube, social bookmarking spaces etc.
What these distributed spaces enable us to do, is to keep our personality in multiple places at the same time and variate our presence in different modalities.

The result of keeping distributed self increases likelihood that my external knowlege, my artifacts, my meanings, my activity patterns will be noticed, modified and duplicated.

Keeping distributed self keeps us in touch with different communities.

Being simultaneously in different communities enables us to bring information across the borders of the communities, initating semiosis, enabling us to constantly create new knowledge.

The maintenance of distributed self has also become external – we tend to feed together our distributed spaces into aggregators or weblogs in order to feel as a whole and observe our external presence. In these places (hubs) where our distributed knowledge meets again, we propagate ourselves as the connectors between the communities.

Can we create in these spaces as well? If we mix our distributed self with the knowledge of our community members (like in microblogging feeds of Jaiku or Twitter), these mashed feeds may work as triggers for writing new blog entries. They enable us to access knowledge community-wise and transfer it to our other community spaces.

The social media starfish is a good representation of our distributed self. Another idea of digital and distributed self is here.

There is also an article by Stanton Wortham about distributed self – The Heterogeneously Distributed Self (2007)
Journal of Constructivist Psychology, Volume 12, Issue 2, March 1999, pages 153-172.

Article explains that heterogeneous distribution can be applied to the self.
The self is heterogeneously distributed because a coherent self emerges from the interconnection of structures of diverse sorts, which together facilitate the experience and manifestation of a coherent identity.

Performative account of self: self emerges when person repeatedly adopts characteristic positions, with respect of others and within recognizable cultural patterns, in everyday social action (Butler, 1990).

The author suggests locating self in several different types of structures, including performative, psychological and other patterns.

For example the author writes how there will be interaction between our past and present self in autobiographical narratives.

This makes me think, if we reflect in weblog – do we also talk with our past and present self in order to create some coherence?


spreading of textual variation (experiment)

October 28, 2007

Anatole Pierre Fuksas invited me to participate in this experiment: Mutating Genre Meme (more likely: ‘ecology of emerging new memes’)


Copy the questions, and before answering them, you may modify them in a limited way, carrying out no more than two of these operations:

* You can leave them exactly as is.

* You can delete any one question.

* You can mutate either the genre, medium, or subgenre of any one question. For instance, you could change “The best time travel novel in SF/Fantasy is…” to “The best time travel novel in Westerns is…”, or “The best time travel movie in SF/Fantasy is…”, or “The best romance novel in SF/Fantasy is…”.

* You can add a completely new question of your choice to the end of the list, as long as it is still in the form “The best [subgenre] in [genre] is…”.

* You must have at least one question in your set, or you’ve gone extinct, and you must be able to answer it yourself, or you’re not viable.

Then answer your possibly mutant set of questions. Please do include a link back to the blog you got them from, to simplify tracing the ancestry, and include these instructions.

Finally, pass it along to any number of your fellow bloggers. Remember, though, your success as a Darwinian replicator is going to be measured by the propagation of your variants, which is going to be a function of both the interest your well-honed questions generate and the number of successful attempts at reproducing them.

My great-grandparent is A Blog Around the Clock.

My grandparent is Belgrad and Beyond.

My parent is The Ecology of the Novel.

The best abstract painting in european art is:
Black square by Kazimir Malevich

The best mindpushing novel in pseudoethnography is:
The Art of Dreaming by C.Castaneda.

The best psychological film in science fiction is:
Stalker by Andrei Tarkovski.

Passing it to:
Pirkko Hyvonen
George Siemens
Martin Lindner


R. Dawkins has written: people could view many cultural entities as replicators, generally replicating through exposure to humans, who have evolved as efficient (though not perfect) copiers of information and behaviour.

Does the information have to be molecular at all? Memes. This is not something that I’ve ever wanted to push as a theory of human culture, but I originally proposed it as a kind of… almost an anti-gene, to make the point that Darwinism requires accurate replicators with phenotypic power, but they don’t necessarily have to be genes.

Memes do not always get copied perfectly, and might indeed become refined, combined or otherwise modified with other ideas, resulting in new memes. These memes may themselves prove more (or less) efficient replicators than their predecessors, thus providing a framework for an hypothesis of cultural evolution, analogous to the theory of biological evolution based on genes.

Wikipedia says: The meme as a unit simply provides a convenient way of discussing “a piece of thought copied from person to person”.

Charles J. Lumsden and Edward Osborne Wilson published a theory of gene/culture co-evolution in the book Genes, Mind, and Culture: The Coevolutionary Process. They argued that the fundamental biological units of culture must correspond to neuronal networks that function as nodes of semantic memory. Wilson later adopted the term meme as the best existing name for the fundamental unit of cultural inheritance.

Memes spread by the behaviors that they generate in their hosts. Memes have as an important characteristic their propagation through imitation

Memetic drift refers to the process of a meme changing as it replicates between one person to another.
Memetic drift increases when meme transmission occurs with variations.
Generations of hosts pass on these changes in the phenotype. Folk tales and myths often become embellished in the retelling to make them more memorable or more appropriate and therefore more impressed listeners have a greater likelihood of retelling them, complete with accumulating embellishments that may serve to modify human behavior.
Memes can move from one propagation medium to another (more efficient) one.

Very few memes show strong memetic inertia (the characteristic of a meme to manifest in the same way and to have the same impact regardless of who receives or transmits the meme).

Cultures can retain memes that once served a purpose during one epoch or era as vestigial memes.

So basically the meme talk is about spreading neuronal networks that function as nodes of semantic memory through action variables we can observe in behavior, or narrative variables which convey the meme in phenotype? Basically, spreading happens through some viewed action, listened narration or artifact mediation, when ideas were made external, toolisized, embedded to the environment for deliberate spreadout or as a by-product of our own activities.
That is well in line with what i think happens at any of our action – in order to use part of the environment for our actions, we need to make it our own, by toolisizing some of its objects through identifying some of their existence as affording our actions and emotions (basically during the neural processing we internalize ‘part of the objects’ from the environment that we may then name affordances, however if we do so, these affordances may be viewed through our behaved actions, narrations and artifacts by other people evoking similar neural patterns mirroring actions and emotions they see).

Meme is a content or meaning part, while meme propagation happens by decoding the meaning from mediation tools/artifacts?

Anatole thinks that ‘memes’ have to be eventually defined as emergent features instead of units of information residing in the brain. Ramifications of descriptions based on multiple potential affordances of described features are responsible of massive textual variation. Textual plasticity found both in oral and in literary traditions, from papyri to manuscript, from print to digital media supporting shared meaning based on collaboration, corroborate the idea that narratives have to be addressed as emergent features, encoded by means of symbolic systems and referring to perceptual events, actions and emotional correlates.

Do i get it right that in narrated text or written artifact we represent a set of affordances that emerged in action/emotion activation for ourselves (we simultaneously embodied part of objects for ourselves and inevitably embedded these embodied perspectives to in these objects). But if someone picks these perspectives (affordances) up from narrated texts, they will see these embedded perspectives (affordances) as correlates of their own embodied action/emotion of the similar kind. Of course that may not be exactly the same perspective (set of affordances) at all, what they have embodied. So it changes the affordances as they propagate them.

Question is, if meme is more like the meaning, while propagation is more like the action potentiality (affordances), we can imagine that in each person the whole reconstruction of the meaning will take place (like diSessa and Hammer etc. have explained will happen when people build up self-explanations of phenomena) rather than the meaning framework (or meme) was transplanted in them (like there are ideas of Stella Vosiniadou). How will these affordances (that are action/emotion related only?) propagate complicated meanings like some Pythagoras school of thinking?
———————- much of trying to understand the background of the experiment.
It is not so easy to write those following the rule: The best [subgenre] in [genre] is…!
I got in trouble with the genre and medium, especially the medium definition 🙂 – i think the medium is the tool where we have defined its affordances (novel – mindpushing), the thing what we spread via this medium is the emotion, action or meaning. Why i need to define genre and subgenre..for me they feel like affordances and i cannot define them when i want to propagate my action potentials, emotions or meanings what relate with these action potentials. The classification has no sense for me in this case, it is something we humans do as a game – classification. I never saw any need for that to propagate my embedded action potentials and emotions. Maybe? for systematic meaning propagation it has some sense? Donno…


Mapping Hybrid Ecology ideas rev.1

October 12, 2007

Since we are thinking to start some really interesting brainstorming session about Hybrid Ecologies in augmented reality in Tallinn I started some ideamapping of certain thoughts.


Several community-activities with new technologies can be built upon the relationships between real spaces/objects, people and meanings:
Space with dynamically embedded meanings (eg. spoken narratives, movement) entails action potentials. The embodiment of these action potentials is a process of coupling affordances evoked by internal imagination and goals in one hand, and the perceived and culturally predefined external affordances from the environment, on the other hand. As a result, this space becomes into a place for new interpretors and starts triggering activities. Activities in places are mediated by cognitive tools (eg. language) or physical tools (eg. body movement).

Space geotagged with meanings as media artifacts (eg. textual narratives or images), which were embedded by ICT tools (mobiles, computers, pdas) or manually (etc. graffit, signs) entails action potentials. This space becomes into an augmented place for real and virtual activities (eg. blogging geotagged itinerary). These activities may be simulatneously mediated by cognitive tools (eg. language) or body (eg. movement) and ICT tools (mobiles, computers, pdas) with interfaces (?) (eg. social software like Flickr, Youtube etc.).

Activities in augmented reality are hybrid – interfaces may theatrialize certain action potentials between real and virtual sides of the realm, intentionally activated internal action potentials may be embedded through virtual interfaces to the spaces toolisizing them.

The thoughts, what might be interesting to study, are still to be elaborated. But i believe in to the interplay between three groups: actors, activity patterns and meanings in augmented places, thus, determining interesting ecosystems.


Here is the initial seed from Anatole Pierre Fuksas:

Storytelling and Hybrid Ecologies in the Age of Social Networking and Locative Media

Philology and criticism usually apply to literary works that have been written and published or literary systems as actual genres. That is, literary studies generally focus on past or present state of the art but they hardly offer predictions, prefiguring forms that will play a role into the future development of cultural landscapes. Making a remarkable exception in respect to the norm, the present contribution aims to forecast potential development in storytelling based on locative media. That is, as part of a more general inquiry on the Ecology of the Novel, it will investigate potential literary applications based on Global Positioning System (GPS), Geographic Information System (GIS) or similar geotagging standards.

In a few years narrative artists and storytellers’ communities will very likely write or tape stories that will be broadcasted by locative media mining 2.0 websites for contents to delievered by Location-based media on GPS or GIS enabled portable wireless devices. Textual narratives as podcasted stories will will invade laptop computers and mobile phones, providing readers and listeners with pertinent references or analogical interferences aimed to enriching natural environments. Presences triggered by the mirror matching of references entailed by symbolically encoded narratives, both in audio and written text formats, will infest urban and rural environments, forests and deserts, islands and hills, mountains and beaches, enhance the sensory experience of perceived landscapes.

So, questions arise. What formats may be forecasted as the standards ones when it will come to the implementation of socially shared narrative art with locative tagging? Will these new narrative standards reshape interactions between subjects and environments? Will coded and shared hybridation based on narrative contents reshape perception of landscapes? Moreover, while providing a permanently operative level of interaction between narrative contents and natural environments, will geotagged stories play crucial role in the literary system? Will eventually the novel be doomed to extintion because of the rise of socially-networked locative narratives? Rather, will the novel outlast this technological revolution too?

See Storied navigation from MIT medialab
i had a talk yesterday aslo with Mauri Kaippainen. He showed me some interesting thoughts where he used Neisser’s framework for describing the situation in hybrid systems.

neisser collaborative

It is notable that Neisser (1994) elaborated Gibson‘s concept of affordance and distinguished three perceptual modes:
– Direct perception/action, which enables us to perceive and act effectively on the local environment;
– Interpersonal perception/reactivity, which underlies our immediate social interactions with other human beings, and;
– Representation/recognition, by which we identify and respond appropriately to familiar objects and situations.

We discussed what the exploration would entail, and can the participants of the community directly view each others’ exploration or is it mediated through the narrative/visual meanings as emergent patterns in the interface.

For me the ‘object’ on the figure is too general, and it does not separate the real objects in the environment and the ‘meanings’ that are tagged to the object. On my opinion these need to be looked together but also as separate things.


My talk with Anatole:

A:If I get you right, you maintain that interaction with objects imply externalization of feelings, emotions, whatever, and the so called meaning the performer attach to the object while perceiving or affording it is somewhat transmitted to the object, that changes into a tool the performer uses to cope with the environment.

K: I assume that i may have internally an emotion or intention (i refer to the meanings or intentions that are initiated inside me as novel, creative), which i cannot express or do without the use of mediation to the external world (to someone certain, to someone abstract). Thus, i must embed this emotion or this action potential to the environment picking something, an object in the environment, or creating an object that didnt exist and attaching my action potentials or emotional cues to this object so that it will become into a tool that helps me to express this emotion/do this action, or to leave this emotion/this action potentiality as an invitation to the environment. This gives the objects i use as mediators a dimension where they become tools. But at the same time they don’t cease to be objects in another dimension.
My understanding is based on the emboding of objects by means of potential or actual affordances.

My understanding involves also the embodiment. I see it as shifting something that is initially left in the environment to some object by someone internal in order to evoke emotions or trigger actions. I cannot use any tools (that the environmental objects with affordances i perceive) are without partly embodying them, extracting something from the object (this something i see as the perspective with affordances), imagining it, initiating some neural processes that lead to my emotions and actions, and then i am able of using this object as a tool to perform action.

When i have used some tool earlier, later this embodied feeling of using the tool with the affordances can serve instead of tool.

Then there is also a possiblity, that i don’t do this embodiment directly by touching the things, but i may observe the others doing it/talking of it.
Or i may read the others having used it. In the last case i somewhat create imaginary tool with affordances, which serves as real tool for triggering my actions and i can use the imaginary tool and do what i wanted to do. (eg. i imagine streets to know where to go, and i can explain it also by using this imagined tool)

A: So, human anatomy is crucial as the shape of the object is and certainly Human phisiology is perturbated by affordances, hence performers can feel like affording a book just by thinking of it or talking about it (mirror neurons etc…).

K: yes..human anatomy and perceptional organs?

A: Now, the actual affordance of an object …

K: i dont believe the object has an actual affordance, it may have many affordance perspectives only due to different actors who have embedded/omitted some of their emotions/action potentials to the object

A:… may turbate the status of the afforded object and the change makes sense to me in terms it determines subsequently different assessment of the landscape it belongs to.

K: i cannot understand word turbate
since i disagree the first part it is hard to get the point of second part

A: I’d redefine the idea of externalizing feelings by actually affording objects in different terms.

K: yes, exactly – omitting internal affordances which we imagine as necessary prerequisties for actions/emotions to the objects in external world.
However i also believe it is kind of coupling process. The objects due to our perception, earier experiences, cultural background, embodied action potentials of these objects will always sort of extend some properties to us, that makes us to pick these objects, feel as if there are paired affordances for our action. So for me at both sides there is matching between imagined affordances.
Is it the same what you think with previous sentence of turbating?

A: That is I’d say goal planned actions so to be defined as purposeful and planned at all have to entail emotional correlates (Bechara and Damasio’s studies I quoted and I attach, one of them I have in paper, so send you the reference and another one I just got the abstract).

K: logically this sentence is not argumented here. But in principle i believe they come as pairs.

A: So, emotions are not externalized, they are simply part of the decision making process…

K: there are some emotions that precede any intentions, and there are some emotions that procede intentions or are simultaneous with emotions

A:… that is they are crucial and completely embedded into the action planning strategy.

K: at least two last versions of emotions are
but some emotions exist per se, not because of intentions
creative emotions often have no goal but emotion

A: To a certain extent, perception triggers action potential and emotional correlates at the same time.

K: i agree

A: So, in order to ‘get’ the emotion driving somebody else into, say, leaving a book into a bookshop, you may be able to decode it from CLUES he left WHILE interacting with the object, that is handling, manually affording it.

K: Yes i believe you attach the emotional potentials to the tools when you actually embed the affordances of action.
But for pure emotion to evoke emotion, do you need the action/goal things in between or can you also embed to some objects, afford pure emotion?

A: Dunno if this fixes the point and sets us on the same page, but I think it may help you with your externalizing struggle

K: for me we are at the same ground, the question was more in my use of words: embedd, externalise, omit, embody is somewhat nice word for himans to take something as part of their body parts, but for me actors and objects do not feel that different. For example trees feel me same as humans. Or stones. Or houses. Or water. So i could somewhat imagine they have some goal-directed embodiment as well.

For example there are studies in the north how trees transmit fear. Or there are many studies of water having structural kind of memory. If to think a bit more open way further, you can see my struggle with embodiment isnt so simple.


intersubjectivity from embodied simulation

July 5, 2007

I have had interests towards intersubjectivity as a phenomenon for quite some time.
Reading the works of Gallese i found interesting explanations that relate intersubjectivity with mirroring systems of our brain, and simulation processes:

The mirror matching system: Ashared manifold for intersubjectivity
Vittorio Gallese, Pier Francesco Ferrari, and Maria Alessandra Umiltà

Simulation theory in fact holds that we understand others’ thoughts by pretendingto be in their “mental shoes,” and by using our own mind/body as a model for the minds of others (Gallese & Goldman 1998; Goldman 1989; Gordon 1986; Harris 1989).

Preliminary results suggest that a mirror matching system could be at the basis of our capacity to perceive in a meaningful way, not only the actions, but also the sensations and the emotions of others (see Gallese 2001).

In conclusion, these recent findings suggest that a neural matching system is present also in a variety of apparently non-motor-related human brain structures. Thus, different simulation mechanisms are applied in different domains, being sustained by a mirror-matching, dual-mode of operation (action-driven and perception-driven) of given brain structures. We propose that such simulation mechanisms may constitute altogether a shared manifold of intersubjectivity (see Gallese 2001).

Subitted to JAPA
Vittorio Gallese / Paolo Migone / Morris N. Eagle*

Such personal body-related experiential knowledge enables our intentional attunement with others, which in turn constitutes a shared manifold of intersubjectivity.

This we-centric space allows us to understand the actions performed by others, and to decode the emotions and sensations they experience. When observing others we do not just see an action, an emotion, or a sensation. Side by side with the sensory description of the observed social stimuli, internal representations of the body states associated with these actions, emotions, and sensations are evoked in the observer, as if he/she would be doing a similar action or experiencing a similar emotion or sensation.

Fonagy & Target (1996a, 1996b, 2000) have shown within the context of their studies on self reflective function, the ability on the part of the mother to think and react as correctly as possible to the infant’s mental state (his/her intentions, affect states, etc.) will allow the infant to build the ability to understand his/her own mental states as well as those of others (see also Fonagy et al., 2002).

This is how Wood et al., 1976 originally described what scaffolding is…comprehending mutually each other’s intentions and mindset.

The shared blended space enables the social bootstrapping of cognitive and affective development because it provides an incredibly powerful tool to detect and incorporate coherence, regularity, and predictability in the course of the interactions of the individual with the environment. The shared space is paralleled by perspectival spaces defined by the establishment of the capacity to distinguish self from other, as long as self-control develops.

The shared intersubjective space doesn’t disappear. It progressively acquires a different role: to provide the self with the capacity to simultaneously entertain self-other identity and difference. Once the crucial bonds with the world of others are established, this space carries over to the adult conceptual faculty of socially mapping sameness and difference (ì am a different subject).

Given the shared sub-personal neural mapping between what is acted and what is perceived constituted by mirror neurons the action model can also be used to predict the consequences of actions performed by others. Both predictions (of our actions and of others’ actions) are instantiations of embodied simulation, that is, modeling processes.

Embodied simulation automatically establishes a direct experiential link between agent and observer, in that both are underpinned by the same neural substrate.

Can a machine be an agent?

The agent parameter must be filled.
Indeed, not all kinds of agents will do. The brain imaging experiment on communicative actions shows that only stimuli consistent with or closely related to the observeris’ behavioral repertoire are effective in activating the mirror neuron system for actions (Buccino et al., 2004).

It has been proposed that simulation process may constitute a basic level of experiential understanding, a level that does not entail the explicit use of any theory or declarative representation (see Gallese et al., 2004; Gallese, 2004, 2005).

Our seemingly effortless capacity to conceive of the acting bodies inhabiting our social world as persons like us depends on the constitution of a shared meaningful interpersonal space. This shared manifold (see Gallese, 2001, 2003a, 2003b, 2004, 2005) can be characterized at the functional level as embodied simulation, a specific mechanism, likely constituting a basic functional feature by means of which our brain/body system models its interactions with the world.

When we confront the intentional behavior of others, embodied simulation generates a specific
phenomenal state of intentional attunement. This phenomenal state generates a peculiar quality of familiarity with other individuals, produced by the collapse of the others’ ́intentions into the observers’ ones.

The gap between the two perspectives is bridged by the way the intentional relation is functionally mapped at the neural-body level. Any intentional relation can be mapped as a relation between a subject and an object.

It might well be the case that embodied simulation scaffolds the propositional, language-mediated mechanism.

Simulation is a functional process that possesses a certain representational content, typically focusing on possible states of its target object.

I still struggle with the problem what is the source of intersubjectivity if we cannot perceive the actor directly. The theoretical studies about what neural processes take place when i look at the objects and artifacts around without the actor’s presence and interaction with them seem to be out of my knowing..

forms of intersubjectivity

But yet, here is the image what i constructed about a year ago when searching and thinking about intersubjectivity. It seems to me some of the ‘artifact’s knowledge and intentions’ are missing from the picture of embodied simulation.

There are studies from neuropsychology about the transfer from one kind of information (eg. sound, images of facial expressions) into the intentional activation of mirror neurons. For example if the monkey hears someone to crack the nut, it can assume that there is some food around..or if someone looks at the images of facial expressions the observer would be experiencing a small dose of the emotion corresponding to the observed person’s facial expression (Ekman, 1993, 1998; Ekman & Davidson, 1994). And then mirror neurons get activated. Do artifacts in action (either cognitive or manual application) trigger the mirror neurons?

Supporting the theory of ecological narratives with the mirroring studies would need some neuroimaging proof.

Anatole Fuksas is pointing to some possible sources. I would like to know more about the studies of silent reading in relation to activating inner action with mirror neurons.

Flöel, A. – Ellger, T. – Breitenstein, C. – Knecht, S. 2003
Language perception activates the hand motor cortex: implications for motor theories of speech perception, in «European Journal of Neuroscience»18: 3: 704-708.

The theory predicts that listening to the ‘gestures’ that compose spoken language should activate an extended articulatory and manual action-perception network. To examine this hypothesis, we assessed the effects of language on cortical excitability of the hand muscle representation by transcranial magnetic stimulation. We found the hand motor system to be activated by linguistic tasks, most notably pure linguistic perception, but not by auditory or visuospatial processing.

I could not access full paper so it is a bit unclear, but it does not say they used narrative written texts.

Watkins, K. E. – Strafella, A. P. – Paus, T. 2003
Seeing and hearing speech excites the motor system involved in speech production, in «Neuropsychologia» 41: 989-994.

In this experiment the size of the motor-evoked potentials was compared under the following conditions: listening to speech, listening to non-verbal sounds, viewing speech-related lip movements, and viewing eye and brow movements.
Thus, text reading was not tested.

Watkins, K. E. – Paus, T. 2004
Modulation of motor excitability during speech perception: the role of Broca’s area, in «Journal of Cognitive Neuroscience» 16:6: 978-987.

During auditory speech perception, there is increased excitability of motor system underlying speech production and that this increase is significantly correlated with activity in the posterior part of the left inferior frontal gyrus (Broca’s area).

Again, it does not support the ideas of reading text and mirror neuron activation.

Then, action-related knowledge can be retrieved not only by visual or auditory perception, but even by language, that is by sentences actually describing actions (Watkins and colleagues 2003, Flöel and colleagues 2003, Watkins and Paus 2004, Tettamanti and colleagues 2005).
Indeed, the mediated perceptions of narrative actions might induce motor facilitation, triggering action potential as the planning, the observation or the the auditory clues associated to given actions do.
Since mediated visual stimuli actually trigger ideomotor actions, some similar degree of ideomotricity may be entailed by the textual processing of narratives, both through direct listening to stories verbalized aloud and internal verbalization after silent reading. If so, textual perception and narrative action might share a common coding mechanism, as perception and action do. That is, recognition of narrative action through the pragmatical flow of the text should be supported by the activity of a mirror matching mechanism.

The Ecological Theory of the Novel mantains that perception and action are tightly connected through the narrative flow of the novel. The description of the setting features verbs, nouns and adjectives actually referring to perceptive events, sensory-related properties and body part-specific affordances of items in order to trigger action potential. Narrative action, entailed by motor, sensory and action-related verbs referring to actual affordance, exerts the potential entailed by the described items. Since references to states of mind, emotions, evaluations are seldom independent from perceptual and action-related events, a general network may subserve processing of both body part-specific and general aspecific events, effectors, attributes.

What is the neural mechanism of semiotic processes? Basically, there is similar kind of perspective awareness happening. However, beyond perspective awareness the person who sees the others or some people in action with objects, must also have its own intentional structure – clear grasp of own and unfamiliar differences. Can we see the same intentional structure in written narratives or even narrative images?



intersubjective signs in space

January 25, 2007

I found a nice comment from Pasta and Vinegar blog about game-signs.

intersubjectivity as signs in space

This made me think of one of my earlier thoughts about intersubjectivity in space.
I had my venia legendi recently in Tallinn University about different aspects of intersubjectivity where the similar aspect was brought out how signs may be working in space as some kind of intersubjective knowledge, that each interpreter of the sign will use for certain kind of mutual activities and regulation, nevertheless in which context they are performing.

intersubjectivity image

If to think one step further from this, to our Taggin’ Tallinn community games, then maybe it is a solution to m-communities to use such signs as the indicator of the presence of some activities in real space, that would then provoke different meaning-building actions getting unified under certain label. Maybe these signs would be distributed by phones or can be seen in real space as well.