2 Dec 2010 SRG Meeting Notes
Attending: Georgios, Daniel, Dick, Luuk, Jon, Taku
Taku begins with a comment on the titles of our papers, “When you’re looking through NIME papers, the titles are rather boring.”
He would like us to think of less banal paper titles to make our work stand out a bit more from the bevy of more techy NIME papers. The titles we have now are good for subtitles, but they aren’t enough to ignite that spark of interest.
From there we went around the table, taking the papers one at a time.
Penta-Digit Music // Daniel
The first question Taku and Dick have about this paper is: What is the premise? Daniel’s response is that it is a paper dealing with mapping without calling it mapping by taking a bottom-up, musician-centric approach. The paper describes Daniel’s setup, where one instrument is used as a remapping system for the rest. It is a very cartesian system but there is an interlinking system going on. The left hand changes the behavior of the right hand on the fly.
In the abstract, Daniel explains this as a “fresh look”. Taku suggests another two lines explaining what is the fresh part. Daniel explains that he wants to explain things like mapping in musical terms. This paper comes from practice. Which is nice. From the musician’s perspective. But this needs to be laid out clearly in the abstract. For example: “This paper presents a collection of instrumental devices” … approaches mapping from a practitioner view.
We discussed what the best format for this paper would be. One line of logic that Dick suggests: Daniel has a bunch of devices and they constitute a class. What they do constitutes a certain method for dealing with objects of that class. The method is strong because there is a class of devices. The method is not only instrumental but also teachable.
An important point to take away from this paper is that we can extract a teachable practice from a musical pursuit.
iPhone cracklebox // Jon
Taku and Dick both think the framing here is essential to the success of the paper. What is the thesis?
Jon’s first thoughts on what the thesis is: Taking a physical and artistic object and abstracting it to apply to a virtual platform.
For Daniel, through discussions he’s had independently with Jon, the dynamic mapping approach and parabolic edge behavior was most interesting. So the question is how these things can fit within a larger thesis?
For the abstract itself, Taku suggests taking out the first sentence, and paring down the section on context and dive deeper into the development process and details. Both the iPhone and Cracklebox already have a story behind them. A new story doesn’t need to be written. The context is already there.
For Jon the issue is taking a physical interactive object and adapting it to a platform, which is a framework for objects and has its own context and characteristics. Dick doesn’t like the word virtual, due to its myriad connotations. He suggests to instead use the term “on screen”. Taku suggests Jon look into something Joel had written about “physical handles on virtual models”.
Jon sees the element of tangible computing here. Where do you draw the line on a platform like the iPhone between the tangible aspect and the virtual aspect. Especially when trying to emulate a tangible object like the cracklebox. Where are the lines between tangible and virtual drawn?
Dick asks: “What part of the cracklebox is the paper taking into consideration?” Because the cracklebox is many things. It’s an interactive sound object. It’s a piece of art which addresses the connection between the human body and the underlying circuits. The paper must be very specific on how the cracklebox is being taken into context. Jon is looking at it primarily as a musical experience. His analysis takes place inside the interactive experience of the original object. The interactive paradigm of the cracklebox, which is a process of exploration, of navigating, of finding places you like and staying there. An unstable sound source that is chaotic by nature of the environment and the physiology of the person operating it – via circumstantial and environmental forces. The artistic connotations of connecting the human body to the underlying circuits are lost on the iPhone platform.
Taku suggests to Include these ideas in the keywords: the original definition of what the cracklebox is dealing with. As an art object.
Jon is trying to replicate the whole experience of the cracklebox. He tried a literal translation, but soon realized that the context of the iPhone, both technically and conceptually, don’t allow that
Juggling Balls // Luuk
Luuk wants a good name for the balls.
Dick suggests that what is most relevant here is the development of the balls: the restrictions imposed by Tom and the jugglers, and how it was approached. Dick also suggests replacing the term “algorithmic composition” with “pattern-based composition”.
The paper can lay out the criteria of what is important to the jugglers.
We were presented with a challenge of what the minimum of the interaction should be.
The Wii Years // Georgios
Georgios began by describing his interviews of Alex and Richard Scott. With Alex he analyzed technical and ergonomic issues. Pressure was important. Richard Scott was only using the Wii-motes as wireless buttons.
Dick asks the question: Why is he not using acceleration?
Georgios says it’s because he’s fine with the motion detection of the lightning. Dick thinks that if interviews with Richard Scott are included, they should cover why he’s not using acceleration.
Georgios considered discussing ergonomics and physical design of the Wii controllers in his paper. Eg. the buttons are quiet, free motion (wireless), essentially with 5 degrees of freedom. Daniel suggest he be clear about what those degrees of freedom are.
He would also talk about the problems with bluetooth in a performance context. The Wii is not useful for pitches. The problems with detecting force peaks. How to deal with the gravitational offsets of the accelerometers. Georgios found a group of residents (dancers) in 2007 who found solutions for all of these problems. Funny enough: Alex, Richard, and Frank haven’t found solutions to these problems.
Taku questions to the format of the paper and tries to nail down the premise of the paper. Do we want to create a how-to of designing a Wii instrument? STEIM has been using the Wiis for three years. On the surface level there are practical issues and there is also a question of why we have been supporting artists like Alex who use the Wii controllers. What is the cohesiveness here? The cohesive element will be the premise of the paper.
Dick suggests that, as a researcher, Georgios use the facts to support his own conclusions, and to think about framing the paper through those conclusions. Georgios thinks nobody will be using Wiis again in the future. Even Alex conceded that the Wii was just a game controller.
Daniel thinks that it’s easy to say that in retrospect, but for Georgios, he has to focus on the mindset Alex was in when he first discovered the Wii controllers and a whole new world of musical expression opened up to him.
What does the Wii represent? Why did we use them at STEIM? Everyone eventually agrees that, at its core, it’s not about the Wii itself, but about acceleration mapping. About using the Wii as a prototyping tool for acceleration-based instruments. Accelerometers were unknown to the mass populace before the Wiis/iPhone, they were expensive, and the Wiis brought acceleration instruments into the forefront.
STEIM used Wii controllers because it’s a quick accelerometer-based system. By analyzing the projects done at STEIM you can analyze the uses of acceleration in instruments. With Alex, while using the Wiis he learned what it was he wanted. Now many performers at STEIM, like Alex, or Andreas Otto, are moving towards dedicated wireless accelerometer sy
Perceptual Reverb Instrument // Berit
Dick really likes that this paper is focused on the listening experience. He thinks all of our papers should try to focus on experience rather than start with the technical stuff. Eg. listening experience, interaction experience, performance experience. He also questions the use of the word “cognitive” in the abstract. Otherwise, looks good.
Making Instruments from Instruments // Everyone, mostly Berit & Jon
We’ll have a meeting next week specifically to discuss this paper.
Taku thinks the title is a little long. Dick thinks it’s missing a good name for what we will call this “performance recording” … some names thrown around were “signature” and “trace”. Taku brings up that mapping sensors has been covered extensively, and automation of parameters via recorded data streams is available in most commercial DAWs. What differentiates our research?
Jon suggests that we are mapping qualities and sections of an annotated and analyzed performance to sound parameters, but questions whether we are directly addressing that aspect of the research in this preliminary experiment.
Daniel brings up that in the abstract we use words like “instruments” and “performances” and “performance gestures”, but we need to be clear about what these words mean in our given context. What Taku thinks is especially interesting is the multi-dimensional nature of the data. The instruments we are looking at are multi-modal interfaces. The way the multiple data streams interact creates a kind of signature.
In our paper we need to be clear about what the goal is and what we are trying to achieve. Is it just research? Is to try extract what a performer does and use it in an interesting way? Daniel posits that it could be a learning tool for artists to understand what they are doing, to understand their instruments and compare themselves to other artists. It’s a way of dealing with multi-dimensional gestures, going back to them and naming them. For Daniel it’s a sort of notational system as well, a kind of figured bass for electroacoustic music.
With this paper and experiment we still need to come to an agreement on the hypothesis. In an earlier discussion, Berit suggested that we are trying to see if the data is musically meaningful on its own. As Jon sees it, the supposition is: there is a way to extract a signature of a performance from data, that there is a way of attributing semantic meaning to the performance in a way that defines it uniquely and sets it up for reuse and comparison.
Taku still isn’t convinced. He asks, why do we want it? What do we want to offer from it?
Extracting “words” from a performance. Giving meaning to performances and gestures in order to understand them.
Building an instrument from a performance signature.
Creating a pool of gestures and sensor utterances that come from artists, creating an “impulse response” for a performer, which can then be combined in performance.
These all pertain to the question of whether the gestural data is musically meaningful, and if so, under what conditions?
To get it into the conference we need a strong point. Taku suggests we distance ourselves from the current experiment a little bit, by defining the context, method, and projection of the experiment.
At this point the meeting ended, but myself and Pinar continued discussing the purpose of this preliminary research. I add it here because I think it’s relevant to next week’s meeting specifically addressing this research paper.
Pinar brought up points from the discussion we had at Kristina’s a few meetings ago, where we came to the conclusion that the key point of this experiment is in devising names, categories, and qualities by which to annotate performance data.
Put another way, when we see something interesting, what do we call it … based on the physical actions which caused the data changes, the way the data looks through visualizations, and the sound which results?
In pattern recognition parlance we are at the feature generation stage, which is a process of creating descriptors, a set of features by which to classify gestural utterances. These features will form the basis of a structure for analyzing a performance based upon gestural data.
Until we have these features and give them names we are forced to speak in a language we don’t know yet.