Media Theory for the 21st Century

January 24, 2008

Opaque Code and Code Movie

Filed under: Discussion — Dr. Dave @ 1:07 am

Something Prof. Raley said on Tuesday about the two kinds of code opacity made me rethink a work I’ve written on a bit: Code Movie 1, at the Electronic Literature Collection vol 1. This isn’t my link for next week; I’ll post that later.

I read this movie as making two statements. One: we perceive different file formats differently (a reading suggested by the author). Two: because the only access to a computer’s insides is through itself, is it possible ever to get a real picture of what happens inside from moment to moment? I’m curious if anyone agrees — particularly with my plot summary of the work, which follows:

An image is placed on a scanner. The “bars” of letters that sweep across it are the hardware inside a scanner that digitize it, represented digitally. The image becomes a set of information (represented by hex). It goes through more processes of storage, transformation, and maybe even editing, until it appears on the screen as a solid block at the end, where it is sent to the monitor to be displayed. The final fade-to-white represents that as we see an image (the white screen), or some information displayed digitally, the actual representation of it in code disappears.

My second reading is, I think, the most obvious one. What we see is only a collection of images of numbers that represent hex codes that represent voltage differences, and many different encodings of voltage differences at that — in RAM, on a hard drive, in the scanner cable, and on the screen. We can’t see into the depth of the system, only its representations of itself to us through its own executions.

The first is to me less obvious and bothers me a little more — the formats in which we store data do affect how we perceive the content. If you decompile the movie, you will find that each number is not just an image of a number, but an actual number stored as a character (the difference is between a JPEG of a six and the numeral 6 — you could extract the numbers from Code Movie and reconstruct the images algorithmically without too much trouble).

I find this second reading more chilling because it’s the author’s reading: Biegelman says in her artist’s statement: “Can we think in a poetics of transcodification between media and file formats?” I find this extremely antihumanist: though far be it from me to say that we can’t interpret this in other ways, what does poetics become when an author’s stated intent is to make a comment about how computers display data?

Advertisements

1 Comment »

  1. Because Beigelman’s piece begins with and ends with the hexadecimal shorthand for the JPEG file’s binary code, I would be more inclined to “read” Code Movie 1 as beginning with the already encoded image and then ending with the code for the computer display’s color space. Hex couldn’t be the pre-scanned image but only a representation, or translation, of it after the scan. I would agree then that the middle section presents some interpretation of the processing of the numbers representing the image.

    I think that it is significant Beigelman chooses the JPEG format for her piece, since it is designed to encode and decode images. JPEG, MP3, MPEG, and other encoding schemes are designed to allow the computer to present comprehensible information to human beings as opposed to say network control protocols that ensure computers understand one another.

    For this case, I think it is important that we have some basic understanding of the what is happening at the algorithmic level. JPEG encoded files are substantially smaller than BMP or PICT files because they use lossy compression rather than directly recording the RGB values of each pixel in the image. Since the human eye is more sensitive to variations in brightness over large areas and not small flashes, the JPEG algorithm first converts the RGB color information (which contains luminance or brightness information for each channel—red, gree, and blue) into a YCrBr color space (which concentrates all the brightness information into the Y channel). You can read about the details for the remainder of the algorithm at http://www.w3.org/Graphics/JPEG/itu-t81.pdf. This process does cause some detail to be lost, but the human eye barely perceives that loss. The resulting color space data is broken into 8bit by 8bit matrices on which a type of Fourier Transformation (a discrete cosine transform) is applied with the ultimate goal of eliminating as much redundant information as possible. In essence the least predictable pixels are left for writing to the JPEG file, and the algorithm when run in reverse “predicts” what the missing values should be. Inevitability the algorithm makes some mistakes, and hence information is irretrievably lost, which gives the name lossy to the compression scheme.

    Dave is right to say that the format in which the data is stored does affect how human beings view it, for in the case of a JPEG file, the digital encoding process cannot capture all the information in the analog light waves. That affect, however, is not one-way, since the way the computer manipulates the image relates to the way the human user views the image. Part of Dave’s concern seems to be that the human user has no direct access to how the computer stores and manipulates that image and is therefore excluded from the creation and representation of the image. I don’t entirely agree with that, but even granting it, the computer would have a similar problem of access. Whether it performs a discrete cosine transform or another type of Fourier Transform is arbitrary, and contingent on a person being the viewer. Indeed, the computer would have no access to the physiological or social level on which the image encoded in binary in its memory is used by the human being. Any opacity would be mutual.

    Rather than making an anti-humanist statement, Beigelman’s call for a poetics of transcodification between media and file formats recognizes that humans need to be aware of the computer as an agent participating in the artistic process.

    Comment by James Pulizzi — January 25, 2008 @ 6:54 pm | Reply


RSS feed for comments on this post. TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Blog at WordPress.com.

%d bloggers like this: