Strathernograms Or Semiotics Of Mixed Metaphors
Spring Break Essay Whstaylorbrigancespring Break Essaywater Shoes
Advertising Analysis Essay
Semiotic Analysis Of Gender In Advertising Essay Midnight
Planning Phd Thesis
Blade Runner Essay Blade Runner Essay Gxart Blade Runner Essay
Semiotic Analysis Essay
Essays Samples Different Types Of Essays Samples Starting From
Gram Positive Gram Negative Short Essay
Example Of An Outline For An Essay Resume Sample Essay Outline
Logical Order Of An Essay
Essay Advertising Ad Analysis Essay An Introduction To
The Knowledge Of Photography
Semiotic Studies Semiotic Analysis Report The Best Perfume
Realism Essay Essay To What Extent Does Realism Differ From
Stand By Me Essay
Essay On Power Of Media Essay
Semiotic Analysis Essay
Animal Farm Essay Animal Farm Essay Gcse English Marked By
Abstract: Film cognitivists have been working on constructing models for human film cognitive processes since the 1950s. While North American cognitivists use a pure cognitive approach, European cognitivists built models from the perspective of semiotics. This paper follows the method of European cognitive film semiotics. Previous European film semiotic models focus on the crucial role of film syntax in meaning making processes based on Noam Chomsky’s generative grammar theories. However, in fact, human cognitive processes are not syntax centered, but develop and exist in invisible networks. By emerging Peircean infinite semiotic models and Ray Jackendoff’s “Parallel Architecture” linguistic model, the new model was built upon the notion that film cognition is based on simultaneous interacting/interfacing among film’s syntactical, iconological, phonological, and semantic underlying formation rules/constraints. Also the model indicates that film cognition processes are non-linear but embedded in complex meaning networks.
Human beings can understand films, despite different degrees of comprehension between a child and a film critic. This ability is called film competence, derived from the concept of linguistic competence referring to the system of linguistic knowledge possessed by native speakers of a language, proposed by Noam Chomsky (Chomsky, 1965). Although there is a significant amount of research on linguistics and film content analysis, we rarely ask this question: how do we comprehend films? North American cognitivists work within a pure cognitive framework by dismissing semiotic approaches, while European cognitivists simulate cognitive science into semiotic framework (Buckland, 2000). This paper follows the tradition of European cognitivists and addresses the question on the processes of film cognition through a cultural semiotic approach. The main object of cognitive film semiotics is to construct unobservable underlying structures shared by all films based on linguistic and semiotic models. I propose that since human beings are symbolic species with the ability of comprehending signs, including native languages, visual or sound messages (signs), we can attempt to simulate knowledge and models from language cognitive processes and apply it to non-linguistic realms (films) to address the question on construction of the model of film cognitive process.
Some researchers who construct models on film syntax found that since the absence of a standard cinematic language, it is implausible to pose a theory of filmic competence. It is true that film cognitive processes are unobservable and therefore, it is impossible to build an exhaustive model on specific syntax of film. This paper, however, goes beyond the specificity of spectator/director comprehension, film syntax, content analysis, and cultural differences, and suggests a parallel, concurrent and inter-subjective model of human film cognition.
The Semiotic Cognition Approach
Semiotic cognition approach, in general, uses language as the modeling system for other “second order” systems that function based on structured rules, such as visual and auditory sign systems, in order to figure out the cognitive processes of “second order” systems. Linguistic cognition has its rich research achievement. The cognition processes of “second order” sign systems, compared to linguistic cognition, has not been explored thoroughly. From the perspective of semiotics, “second order” cultural sign systems (films, paintings, or music) are called language-like systems since they share characteristics of “generative grammar” with human language. According to Chomsky, generative grammar refers to a language system that can generate infinite sentences based on finite grammar structures. The concept of generative grammar was successfully applied to non-linguistic model by Ray Jackendoff in the 1980s in his article A Grammatical Parallel Between Music and Language, which applies knowledge of generative linguistics to music with accommodation and adjustment to music specificity (Jackendoff, 1982). Further, Jackendoff, with his student Neil Cohn, found out that people use a narrative structure to comprehend visual sequences and that the brain engages similar neurocognitive mechanisms to build structure across multiple domains (Cohn, Jackendoff, Holcomb, Kuperberg, 2014). Additionally, in general, the cultural semiotic approach was developed by Yuri Lotman, “Among all these systems, language is the primary modeling system and we apprehend the world by means of the model which language offers. Myth, cultural rules, religion, the language of art and of science are secondary modeling systems. We must therefore also study these semiotic system which, since they lead us to understand the world in a certain way, allow us to speak about it” (Eco, 1990).
This paper follows the method of semiotic cognition that is to model linguistic knowledge to film meaning making. However, It is controversial on the necessity of exploring the film narration from the perspective of semiotics: North American cognitivists who use pure cognitive approaches might pose the question on employment of linguistic/semiotic models to films. David Bordwell, who is an American film theorist undermines the value of semiotic analysis on film cognition in his book Narration in the Fiction Film, “ Why is the employment of linguistic concepts a necessary condition of analyzing filmic narration? Is linguistics presumed to offer a way of subsuming film under a general theory of signification? Or does linguistics offer methods of inquiry which we can adopt? Or is linguistics simply a storehouse of localized and suggestive analogies to cinematic processes?” (Bordwell, 1987). Warren Buckland, as an advocate for film semiotic approach, argues against this notion and proposes that even though applying linguistic model to film analysis will subsume film under the system of signification, the fundamental premise of semiotics is that the who human experience is an interpretive structure of mediated and sustained by signs (Buckland, 2000). By embracing the specificity of film, it is useful and necessary to apply linguistic/semiotic knowledge to film studies. This paper is not trying to demonstrate the necessity of semiotic approach to film theoryin general but opens one possibility of the application of the linguistic cognitive model to film cognition.
Generative Meaning Making Processes are in Networks
Human society is symbol-dependent and our culture is inseparable from the expansive system of symbolic systems, from the primitive way of keeping records by taking knots to human language and then to inter-lined symbolic systems such as arts, films, paintings or music. From a semiotic perspective, human meaning making processes are in symbolic structured ways.
Ferdinand de Saussure argues that word meanings can be mapped by signifier and signified (Figure 1). However, human beings’ real time meaning making is not as simplistic as Saussure’s model. According to Terrence Deacon, who has forwarded influential arguments in symbolic cognition, meanings of a sign are not inherent in its physical existence but are referential relations that this sign is with other signs. This referential and dynamic network relation is also proposed by Martin Irvine, “We not only perceive with our senses in brains adapted to immediate awareness of the world around us, we make mental relations between perceptions and thought, and generate further relations among thoughts connected in vast networks of collectively understood signs spanning many states of time.” Similar critics have been made but in this paper, one point of view that I want to emphasize is that meaning making is not static or in a one-to-one way.
Figure 1: Ferdinand de Saussure’s basic model of semiotics
C.S. Peirce’s infinite semiosis and “semiosphere” provide us a better conceptual model (Figure 2). Peirce claims that signs are consisted of three interrelated parts: a sign, an object (interchangeable with references) and an interpretant, “I define a sign as anything which is so determined by something else, called its object, and so determines an effect upon a person, which effect I call its interpretant, that the later is there by mediately determined by the former” (Peirce, 1931). Figure 2 illustrates the interrelation among these three elements. Sign vehicle can be considered as signifier, such as a written word, an utterance, red traffic light as a sign for stop. The object, is whatever is signified: “stop” is signified by red traffic light. However, the object is not a real-world referent, that is to say “stop” is a mental status, concept. The interpretant is one of the most significant features of Peirce’s semiotic, which decide how sign vehicles interconnect with the mental object and thus make meanings. The connections enables meaning making as a dynamic network, a “semiosphere” expands along with time.
Figure 2: Peirce’s infinite semiosis model: meaning making in networks.
Drawing: Martin Irvine
Instead of semiotic approaches, in linguistic field, that cognitive process is dynamic and intersubjective is proposed by Jackendoff’s parallel architecture (Figure 3) in his article A Parallel Architecture Perspective on Language Processing (Jackendoff, 2007). He first published Parallel Architecture in 1997. Within this architecture, it treats phonology, syntax and semantics as independent generative components that are interlinked by interface rules. It is a model that delineate the real word meaning processing in a more accurate way, “In addition to the theoretical advantages offered by the Parallel Architecture, it lends itself to a direct interpretation in processing terms, in which pieces of structure stored in long-term memory are assembled in working memory, and alternative structures are in competition” (Jackendoff, 2007). The so-called long-term memory and working memory can be understood as the hierarchy of computer memory: Hard drive and RAM. Hard drive stores all the files saved in a computer and all software programs installed. RAM is a repository of all programs and files that you are operating at the moment. Therefore, Hard drive is analogous to human long-term memory and RAM is to our working memory. The meaning making process is to assemble information stored in long-term memory into a momentary working memory. In this context, the long-term memory does not refer to human being’s personal and private long-term memory but refers to our cultural encyclopedia that is shared and learned by our society. The way human being “assemble” information can be understood as making referential relations in Peirce’s infinite semiosis. Moreover, comparing to Peirce’s semiotic model, Parallel Architecture better presents the characteristic of intertextuality/dialogism meaning making, which describes that the meaning structures are already exist (long term memory) before we express or understand certain expressions. It reveals one important point that meaning is not just intrinsic to certain object of relations among certain object but as a presupposition and context underlying our symbolic expressions and comprehension as our “sharing codes”. That is why we could understand each other, not only in language sense but also other structured sign system, such as films.
Figure 3: Ray Jackendoff’s Parallel Architecture in language processing
Learn From Previous Cognitive Semiotic Film Models
Scholars have been paying efforts to constructing semiotic/language knowledge models since 1950s. Among all the early scholars, Christian Metz’s Grande Syntagmatique and Michel Colin’s Grande Syntagmatique Revisited have been the most influential and also controversial linguistic inspired film cognitive model. In this article, to build a model of film cognition from semiotic perspective, it is necessary to review these classic models to learn from their strength and constrains how we can propose a new model based on classics.
Metz attempted to construct a general model of the system underlying all films. A syntagm is a structured group of signs. Basic units of film, Metz argues, are shots. So the grande syntagmatique (Figure 4) proposed an abstract classification of the meaningful possibilities when conjoining shots in narrative films. Also, the structure is highly hierarchical, “this scheme gives us a better outline of the deep structure of the choices that confront the filmmaker for each one of the ‘sequences’ of his film” (Metz, 1974). Metz’s model presents the classificatory structure in successive dichotomies that organize the syntactic organization of scenes. Colin, later, revisited and improved Metz’s model by making direct use of Chomskyan linguistics. However, he didn’t change the features of this model, but only refined details.
Figure 4: Metz’s grande syntagmatique of the image track of narrative film
John Bateman and Karl-Heinrich Schmidt argue in their book Multimodal Film Analysis: How Film Mean, that Metzain grande Syntagmatique received immense criticism but only some of that was valid. From Bateman and Schmidt’s points of view, there are three main streams of criticism toward Metz’s model: robustness, exhaustiveness, and usefulness. Under each category, there are criticisms specifically from different perspectives. In this paper, I want to emphasize three main criticisms to help us build a more robust, exhaustive and useful cognitive film model. First, Metzain models overemphasize syntax/grammar of film narration and neglect the fact that film cognition, meaning making process is concurrent, generative, and real time processes, proceeding in a networked way. Only presenting and organizing underlying syntax is not enough to make sense of our real cognitive processes. During early stage of Metz’s research, he believed that filmic signs are neither arbitrary nor symbolic, but iconic. Later he realized that film codes are not naïve realistic but that of interpretations are intertwined with convention and social context (Bateman and Schmidt, 2012,) Despite, Metzain models failed to present the intertwined relation between syntax and semantics of filmic signs and also the role of other film components such as sound. Second, Metzain models do not indicate layers of abstraction of filmic cognitive processes. In the book The Analysis of Film, Raymond Bellour points out that a possible new version of grande syntagmatique should lay emphasis on this dimension, “On the other hand, the new version of the grande syntagmatique would need to reinforce the level of abstraction in order to stamp out definitively any flattening structural effect, or any descriptive application between code and text thus correcting earlier inadequacies” (Bellour, 2000). The binary hierarchical narrative syntax is not robust enough to describe the filmic cognitive process. Third, Metzain models lay emphasis on spectator’s meaning-making processes, which poses a question: is it possible to build a model that goes beyond the specificity of groups of people? Either film creators or receivers encode and decode film meanings based on the common ground that all of us possess filmic competence, in the same way as we possess linguistic competence. Focusing on spectators’ cognitive processes, instead of a generic model of meaning making will mislead the direction of cognitive film semiotic model construction, Colin thinks that “there is no reason to distinguish between image track and sound track, since they are all in spectator’s mind” (Buckland, 2000). It is true that we do not need to distinguish image track from sound track, but we should acknowledge that it is crucial to realize that sound and image is working concurrently and this concurrency is not spectator specific.
I agree with Buckland on his evaluation on the future direction of film cognition modeling, “Metz and Colin are insufficient in themselves in explaining this processing complexity. Film theory can only benefit from exploring in more detailed these and other recent linguistic models” (Buckland, 2000). There are so many other scholars working on refining grande syntagmatique than Colin and I think we can draw theories on newer linguistic models that can give us fresh perspectives of modeling film cognitive processes. So next, I will introduce the possibility of model construction based on Jackendoff’s Parallel Architecture, and its potential to analyze filmic meaning making through applying it to film The Last Emperor.
Toward a New Cognitive Film Semiotic Model
What we are trying to do is to catch the moment and ask ourselves: how do we make sense of film content? As above, I discussed the linguistic model, Jackendoff’s Parallel Architecture (Figure 3) provide us a model to look at structures beyond syntax but a concurrent structure that phonological, syntactic, iconological, and semantic structures process information simultaneously. However, Jackendoff’s structure does not reveal the fact that our meaning making processes are actually in networks. Peircean infinite Semiosis model (Figure 2), however, provide us with the dynamic, generative, and network-like cognitive processes. Therefore, combining Parallel Architecture and Peircean infinite semiosis, I construct a new cognitive film semiotic model (Figure 5). I added one component, iconological structures, formulation rules, and constraints based on the characteristic of films. The reason why to do the accommodation is inspired by the heated talk on film specificity when film cognitivists trying to apply linguistic models to films. Jackendoff points out in his article A Grammatical Parallel Between Music and Language that applying linguistic model to non-linguistic fields need adjustment according to the specificity of this field, “Previous attempts to apply linguistic methodology to music have proven relatively uninteresting because they attempt a more or less literal translation of linguistic theory into musical terms, for instance looking for musical parts of speech, or deep structures, or transformations, or semantics” (Jackendoff, 1982). In the this model, phonological formation rules refer to formation structures for film soundtracks and conversations; iconological structures are rules based on actors, costumes, makeups, camera, and lightning; syntactic structures are rules for script, direction and film editing. Syntactic structures have been explored by film cognitivists such as Metz explored thoroughly (Figure 4). All these formation mechanics are invisible to us and the only observable things of our cognitive processes are the outcome, films. That is to say, we “utilize” films to organize our thoughts and meaning systems. When we watch films, those images, actors, narrations, or other components remind us of things. The process of “reminding” is a crucial part of film cognition. Also, these “reminding” processes happen within our cultural encyclopedia. We define the cognitive process in one bracket as a unit. The unit can be the cognitive process model for one image of a film, or can be a model for a scene. Since no matter for a static image in a film or a one-minute or even one-hour scene, human cognitive process should follow the similar pattern that grasped by this merged model.
It is hard to simulate human being’s real time cognitive processes in a model because it has to be on the appropriate level of abstraction for developing generalizable and extensible concepts that usefully account for intersubjective, dynamic and network-like activity. In order to know the strength and limitation of the new model, a good way to test its efficiency is to apply it in a real film. So, next, I will apply the model to the film The Last Emperor to have a sense of its working mechanism and possible drawbacks.
Figure 5: A Combined Model of Infinite Semiosis and Parallel Architecture
Diagram adapted from Martin Irvine, Semiotics and Parallel Architecture
An Example: The Last Emperor (1987)
The Last Emperor is an autobiographic film released in 1987 about the last emperor of China, Puyi. It is directed by Bernardo Bertolucci, an Italian director. At the 60th Academy Awards, the film won nine Oscars, including the Best Picture. The film is about Puyi’s life, depicted from his ascent to the throne as a three-year-old boy to his imprisonment and political rehabilitation by the Chinese Communists. Two period of his life time flash back and forward in an intermixed way: (1) from this throne in 1908 and life in the Forbidden City until his exile and became Japanese-supported puppet (up until in his 40s); (2) His later life in prison after his arrival at a prison in the recently established People’s Republic of China as a political prisoner and war criminal. In the end, the film flashes forward to his senior life as a gardener.
As I discussed above, if we see every bracket in Figure 5 as a unit, every unit can be either an image grabbed from a film, or moving scenes lasting for a certain period of time, or even a whole film. So theoretically this model can be used for multiple levels of meaning construction. This is also the reason why I think the input and output is not linear because I see the unit as not a general cultural form (film) but can be used flexibly for either a specific moment, a long shot or a whole film, all of which interpretations are fed within a semantic network. Therefore, we can understand this new model as a model that replaces the modular in Peircean Infinite semiosis, the “three interrelated parts” with “parallel architecture” stack.
So next, I will apply the merged model to the film The Last Emperor in three levels: (1) flashback narration aided by the use of lighting all through the film (2) Two Fuyi chasing scenes (one is in his childhood and the other is in his adulthood) (3) still images: wide angle shots.
The director Bertolucci uses special lighting strategies to form the contrast between Fuyi’s two different period of time: During his childhood, he became an emperor when he was three years old. As the monarchy at the edge of collapse, his life was controlled by monarchy, Chinese newly built republican government and Japanese government. When he was a child, he cannot leave the Forbidden City and he was forced not to see his parents. His only friends were eunuchs and ministries, until he met his English teacher from Britain, “Mr. Johnson”. He learnt English and the outside world (the western world) from Johnson. Through the film’s lighting, a transition from the dark (lack of sunlight) during the emperor’s lonely childhood depiction to more and more natural light on him after meeting Mr. Johnson. After he was expelled from the Forbidden City and Mr. Johnson’s left, Puyi became a puppet for Japan and the light on him becomes less and less again (Figure 6). The “narration” of lightning, from dark lightning to bright and to dimming again, changes along with his life experience. So we can see that the meaning is constructed by the lightning (under iconological formation rules) “interfacing” with plots/narratives (under syntactic formation rules), along with other elements such as soundtrack and actors, as what the model indicates.
Figure 6: lightning, working along with narrative structures, constructs complete meanings for the film. (click the picture to see it bigger: after opening it in a new tab, you could zoom in by changing the scale of your browser, picture itself is in high resolution.)
The second series of scenes that I will analysis are a pair of echo scenes. Puyi is grown up with his amah instead of his mother. So when he was eight years old, he was told that he was old enough to live without his amah. When the emperor Puyi knew that his amah just left, he went out chasing after the sedan that his amah was in (Click here to see the movie, 51:16). The second scene is toward the end of the film, when Puyi was a puppet of Japan, he was told that his own son was born dead (actually was killed by Japanese) and his wife was sent away. He saw from the window that his wife is heading into a car, and then he running downstairs and again, chasing after someone he loves (Click here to see the movie, 3:08:30). Both of two scenes are with the same soundtrack. No matter it was in his childhood or his adulthood, his life was controlled and people whom he loves were all deprived from him. The meaning is constructed through the complex interplay between the soundtrack and the visual, which also follow the pattern that the model suggests. Additionally, it demonstrates that simply syntactic model for film cognition is not enough.
In the last series of scenes, I want to focus on the level of static images grabbed from the film, trying to focus on the interaction between spatial and iconological structures and the film construction. The film The Last Emperor uses a large amount of shots with wide angles. The three screenshots in Figure 7 was the shots after Puyi chasing after his amah. The huge size of the “background” Forbidden City is “over-proportionate” to the size of the emperor Puyi. Here, the film is analogy the physical oversized circumstances to the emperor’s spiritual loneliness and void. Also, when spectators watching this scene, they will combine their knowledge and impressions with the interpretation with emperor’s spiritual life. The scene is a wide angle shot of the Forbidden City, with iconic meanings, such as monarchy and other historical knowledge of China. One of the reasons that I choose this film as an example is to demonstrate that the model is cross-cultural and beyond different degrees of comprehension. The point that I want to emphasize is that although film cognition is in different degrees (people know more Chinese history knowledge and the others) the fact that the scene will always remind people of something that go beyond the visible objects in the film. This “reminding” process, as I proposed before, is a reflection that film meaning making is embedded in the large meaning network instead of a linear fashion, which is also a demonstration that the model has multiple inputs and outputs.
Figure 7: Three sequential screenshots with wide angle shots
Beyond certain series of scenes, the film as a whole interacts with other forms of media, along with time, in a larger network of meaning (cultural encyclopedia). However, the examples can definitely take us from an abstract analysis to a concrete application, and thus to see the inefficiency and strength of the model. From my perspective, the model is comprehensive enough to present the film generative and dialogic cognitive processes. However, some problems and open questions exist: is the model too general and thus not specific enough to show how each of the component (syntactical, semantic, iconological, and phonological structure) works? Is there any necessity to rework certain visual only or narrative sequential only models?
In conclusion, by a cognitive film semiotic approach, and unveiling the fact that human cognitive processes are networks, this paper evaluates the strength and deficiency of the previous film cognitive models and proposes a new model merged from Peircean infinite semiosis and Jackendoff’s linguistic model, Parallel Architecture. The analysis in this short essay is not enough to embrace a perfect model for film cognition. However, it demonstrates the validity of film cognition as concurrently processed by four film structures interfacing/interacting with each other and opens the possibility to insert parallel modules into Peircean semiotics. It is just a start of exploration of film cognitive processes. Further research will be build upon the merged model in the directions of those open questions raised above.
Bateman, John, and Karl-Heinrich Schmidt. “Christian Metz and the Grande Syntagmatique of the Image Track.” Multimodal film analysis: How films mean. Vol. 5. Routledge, (2013).
Bellour, Raymond. “To Segment/ To Analyze (on Gigi).” The analysis of film. Indiana University Press, (2000): 193-196.
Buckland, Warren. “The Cognitive Turn in Film Theory.” The Cognitive Semiotics of Film. Cambridge University Press (2000): 1-26
Bordwell, David. “Narration and Film Form.” Narration in the Fiction Film (1985): 27-146.
Buckland, Warren. “All in the Mind? The Cognitive Status of Film Grammar.” The Cognitive Semiotics of Film. Cambridge University Press (2000): 109-140.
Chomsky, Noam. “Aspects of the theory of syntax Cambridge.” Multilingual Matters: MIT Press (1965).
Cohn, Neil. “You’re a Good Structure, Charlie Brown: The Distribution of Narrative Categories in Comic Strips.” Cognitive science (2014).
Cohn, Neil, et al. “The grammar of visual narrative: Neural evidence for constituent structure in sequential image comprehension.” Neuropsychologia 64 (2014): 63-
Cohn, Neil, et al. “(Pea) nuts and bolts of visual narrative: Structure and meaning in sequential image comprehension.” Cognitive psychology 65.1 (2012): 1-38.
Deacon, Terrence W. “The symbolic species: The co-evolution of the brain and language.” New York: WW Norton&Co (1997).
Jackendoff, Ray, and Fred Lerdahl. “A grammatical parallel between music and language.” Music, mind, and brain. Springer US, (1982): 83-117.
Lotman, Yuri. “Universe of the Mind.” A semiotic theory of culture. London: IB Taurus (1990).
Metz, C., “Film Language: A Semiotics of the cinema.” Oxford University Press and Chicago University Press, Oxford and Chicago. Translated by Michael Taylor (1974).
Peirce, C.S. “What Is a Sign?” The Essential Peirce Selected Philosophical Writings. Vol. 2, Vol. 2,. Bloomington, Ind.; London: Indiana University Press, (1998): 4–10.
Peirce, C.S. The Collected Papers. Volumes 1–6. Eds. Charles Hartshorne and Paul Weiss. Cambridge M.A.: Harvard University Press (1931–36).
Note that persons in screenshots are all the last emperor Puyi, in different life stages.