Week 1: September 9
Discussion of Syllabus
Introduction to Realtime Video
Introduction to Cycling74’s Max
Useful Links:
Assignment:
~/Applications/Max6/patches/docs/tutorial-patchers
Week 2: September 16
Max intro continued
Lab:
Week 3: September 23
Intro to Open Sound Control (OSC)
Introduction to Jamoma
Class Exercise:
Lab:
Define Etude 1: Modify Live | Recorded Video by combining or by applying an effect, provide control parameter(s)
Assignment: Jitter Tutorials 3 - 5
Week 4: September 30
Jamoma
Filters, Effects
Lab: Work on Etude 1: Modify Live | Recorded Video
Assignment: Jitter Tutorials 6 - 10
Week 5: October 7
Advanced Video Processing
Readings:
Week 6: October 21
Extracting Statistics
Tracking value
Related Articles:
Week 7: October 28
Introduction to the Computer Vision Library: cv.jit
Week 8: November 4
Theory
Introduction to MSP
Class exercise: Feedback machine
Week 9: November 11
Class:
Lab: One of the following: Physics or Graphics
Special Topic: Physics
Lab:
Week 10: November 18
Lab:
Special Topic:
Special Topic: Optimization
Week 11: November 25
In-class work period. Each group must meet with the instructors to discuss their final project.
Bench test. Debug.
Tutorials and Recipes:
Week 12: December 2
Lab: In-class work period. Each group must meet with the instructors to discuss their final project.
Rehearse on site. Debug. Optimize.
Week 13: December 9
Final project presentations and critique
Discussion of Syllabus
Introduction to Realtime Video
- Realtime processing
- Responsive media
- Examples: Slitscan, Timespace, broken TV, Particles & Fluids
- Limitations, Strengths
Introduction to Cycling74’s Max
- How to Install Max and Documentation
- The Working Environment
- Max: Objects and Connections
- MAX BASICS...
- Sliders and other forms of input
- Mathematics Operators, Comparisons
- Inlets, Outlets, Flow of execution, Bang
- Routing, Gates and Switches
- Time: Metro
- Iteration: Uzi
- Class exercise: pseudo-code, incrementing counter
- Style: AVOID iterators, indexing into arrays, data structures; USE flow, time-based media
Useful Links:
- New in Max6 Usability Features
- Organized Patching with Trigger
- Functional list of Max objects by type, with links to individual object documentation.
Assignment:
- Max Tutorials 1 - 6
~/Applications/Max6/patches/docs/tutorial-patchers
Week 2: September 16
Max intro continued
- Arguments
- Sub Patches
- lists, zl
- Presets
- Style: modularity, commenting, README, automation, robustness
- Video Matrices, Layers
- What is a Matrix?
- Getting video in: jit.qt.movie, jit.qt.grab
- jit.matrix, jit.unpack, jit.pack
- Displaying video: jit.pwindow
- Jitter Attributes
- Processing example: feedback Jitter Tutorial 18
- Max Tutorials 7 - 13, Jitter Tutorial 1 - 2
Lab:
Week 3: September 23
Intro to Open Sound Control (OSC)
- Enhances your code's organization, documentation, and flexibility
- Open-ended, dynamic, URL-style symbolic naming scheme
- Pattern-matching language to specify multiple recipients of a single message
- High resolution time tags, and high-resolution data
- "Bundles" of messages whose affect must occur simultaneously
- Query system to dynamically determine the capabilities of an OSC server and get documentation
- Biggest problem: no standardization of OSC address patterns, interconnectivity of devices
Introduction to Jamoma
- VIdeo i/o
- Basic video operations
- Effects
Class Exercise:
- Editing and handling OSC messages (prepend, append, sprintf, etc)
Lab:
Define Etude 1: Modify Live | Recorded Video by combining or by applying an effect, provide control parameter(s)
Assignment: Jitter Tutorials 3 - 5
Week 4: September 30
Jamoma
Filters, Effects
Lab: Work on Etude 1: Modify Live | Recorded Video
Assignment: Jitter Tutorials 6 - 10
Week 5: October 7
Advanced Video Processing
- Submatrix
- Scissors and Glue
- Downsampling
- Repositioning
- jit.op
- jit.expr
- example: bfg
- jit.slide
- jit.matrixset
- Present Etude 1: Modify Live | Recorded Video, with parameter
- Live camera in physical space (EV 7.725), Lighting
- Define Etude 2: Gesturally modulate video
Readings:
Week 6: October 21
Extracting Statistics
Tracking value
- Using color space information
- Example: creating a light Theremin
- Value tracking: Jit.findbounds
- Masking techniques
- Frame Differencing
- Background subtraction
- Work on Etude 2: Gesturally modulate video
Related Articles:
Week 7: October 28
Introduction to the Computer Vision Library: cv.jit
- Installation
- Feature extraction: motion
- Feature extraction: shape (morphology): blob, edge, shape
- Learning (Hidden Markov Models), strengths and limitations
- Present Etude 2: Gesturally modulate video
- Extend Etude 2's using cv.jit etc.
- Compare tracking by colour, luminosity, total motion
- Compare tracking by recognizing shape
- Define Etude 3: Modulate one video | sound stream by features from a second video | sound stream
- Example: convolution ripple, particle ripple (Julian Stein)
Week 8: November 4
Theory
- Audio-vision
- Visual-music
- Transmodality
Introduction to MSP
- DAC~ ADC~
- DSP Status: DSP, CPU, GPU and scheduling optimization
- Gain and Monitoring
- Synthesis: ie. Cycle~
- Play~ Buffer~ Sfplay~ Groove~
- real-time Audio Processing
- Granular Synthesis
- Mapping and modulation
Class exercise: Feedback machine
Week 9: November 11
Class:
- Present Final Project: Groups should be ready to pitch their project concept including their strategies for implementation.
Lab: One of the following: Physics or Graphics
Special Topic: Physics
- Newtonian dynamics F = mA, Gravity, Conservation Law
- jit.phys
- Introduction to 3d objects
- Creating simple forms
- The render object
- Importing more complex objects
- Drawing Points and Lines
- Mapping video to a plane
- The slab object
- Compositing in OpenGL
Lab:
- Creating a sound ‘instrument’ based on video
- Work on Etude 3: Modulate one stream by features from a second stream
Week 10: November 18
Lab:
Special Topic:
Special Topic: Optimization
- Scheduling
- Hardware: CPU, GPU, DSP
- Low-level language: Open GL
- Extend via external: compiling C via XCode
- Extend via Gen
- Work on Etude 3
Week 11: November 25
- Present Etude 3
- Optimization
- Open GL
- Physics
In-class work period. Each group must meet with the instructors to discuss their final project.
Bench test. Debug.
Tutorials and Recipes:
- http://cycling74.com/physics/
- Particles: ~/Max6/examples/jitter-examples/gen/jit.gen.particles.maxpat
- Particles: http://www.itaru.org/ilog/2012/11/12/inky-particle/
- Work your framerates
Week 12: December 2
Lab: In-class work period. Each group must meet with the instructors to discuss their final project.
Rehearse on site. Debug. Optimize.
Week 13: December 9
Final project presentations and critique