Computer Games as Gateway to STEM

The Story of “Game Goddess” and ATLAS Ph.D. student Kara Behnke

By Ira G. Liss

Kara Behnke could be on track to become the unofficial “Game Goddess” of the world. Having a passion for computer games – playing them, designing them, teaching others to design them and using them as teaching tools – she is working towards her Ph.D. in Technology, Media and Society at CU-Boulder’s ATLAS Institute.

A Gateway to Science

Behnke believes computer games make excellent gateways to science, technology, engineering and mathematics (STEM) for K-12 students who might not otherwise be attracted to these subjects. “Games are inherently fun. Young people love to play them. So why not use this basic, human desire for fun (an example of positive psychology) to motivate students in the classroom?”

National Science Foundation Programs

As part of her research, Behnke has spent 10 to 15 hours per week in a Longmont high school – part of a two-year program sponsored by the National Science Foundation’s (NSF’s) eCSite program (pronounced “excite”), an acronym standing for “Engaging Computer Science in Traditional Education.” (The NSF is also the sponsor of her ATLAS Ph.D. fellowship.)

Diversity Issues

For decades, the computer science (CS) field – like other science and engineering fields – has had extremely low diversity or representation of ethnicities and gender, particularly women. And yet, to be competitive in a global marketplace, innovation and entrepreneurship depend upon the insights and experiences of all of us – our full range of culture, gender and ethnicity – that together make a healthy society.

To help bring diversity to these fields, the NSF eCSite and GK-12 programs explore ways to bring computer science and computational thinking to existing curricula across multi-disciplines –  including biology, health, art and music. These programs offer the expertise and passion of CS students to K-12 teachers.

Integrating Science in “Non-Science” Disciplines

In these programs, computer science and STEM content are integrated into disciplines where students are already present and motivated. The idea is that these students, given a positive experience with computational thinking, are then more likely to pursue STEM subjects in depth.

In real time, the Microsoft Kinect system scans the movements of  students while software programmed by a student team places their silhouettes in an animated graphic background.

Behnke explained, “This year, I helped an art teacher bring computer science to students at Skyline High School in Longmont. It was a very gratifying experience!”

A Game Interface for Performing Arts

Each spring, Skyline presents a showcase of student arts that include studio arts, 2D and 3D design and dance. Behnke invited students in the afterschool computer science club to collaborate with art students and participate in this art event. She showed students a brief demo of what could be done with the Microsoft Kinect system (a user interface commonly used for games run on the popular Xbox 360) and invited them to “create something cool.” 

Interactive installation designed by students under Behnke’s direction allows participants to see their image instantly transformed through the creative use of the Microsoft Kinect game controller.

"We had 20 to 30 students working on this project. For many CS students, it was the first time they worked on an art project. And for art students, it was the first time they worked with computer technology to make art.”

Kinect software captured the movements of dance students in real time. Their images were scanned and projected on to screens with colorful animated graphics integrated into the imagery. Graphic programs were designed by a team of art and computer science students. Installations were designed and set up in rich environments that invited audience participation.

Multi-Media, Multi-Disciplinary, Multi-Successful

“Their art installations and performances incorporated costume, set, environmental and graphic design along with dance, visual arts, animation and computer science – all produced by  students.” Behnke continued, 

Skyline High School student presents a set design he created in which the multimedia, interactive display and performance will be presented.

“They worked successfully in teams while using technology they had never worked with before. They saw how technology can be a wonderful tool – as creative and expressive as clay or paint. And they experienced how multiple disciplines and media can work together in exciting ways.”

An Untraditional Background

 “I came to computer science from an untraditional background.” Behnke explained. “I knew I wanted to work in the computer game industry when I came to CU. But there was little support for this when I was an undergraduate student. Computer science classes were for engineering students and I was a liberal arts student studying Japanese. The closest program I could find to what I wanted was the ATLAS Technology,Arts and Media (TAM) program (an undergraduate minor and certificate program).

“In TAM, I took an elective taught by ATLAS director John Bennett – Virtual Worlds in Second Life. In the first group to take this class, I was introduced to computer science by working with code, building virtual objects and designing programs – all while playing in Second Life, a 3D virtual world. That experience made a big impression on me.”

Independent Study Becomes Model for New Course

Behnke went on to do an independent study with Bennett designing games for the Xbox 360 using the C# programming language. Their work together became the model for a new course offering, ATLS 4519/5519, Computer Game Development.

“I became a teaching assistant for that new class. It was really satisfying to know I was instrumental – in collaboration with John Bennett – in the creation of that class. It helped affirm my view that designing and using computer games can help students of all ages (myself included!) to learn computer science plus related fields – animation, graphic design, storytelling, coding, user interface and more.”

Her Own Proof of Concept

Behnke may be her own living proof of what she believes, researches and teaches – computer games can lead to deeper learning in the applied sciences. They can bring a diversity of students to fields that currently have very limited diversity. 

Using our natural, inherent desire for fun and play, computer games can help young people discover science, mathematics and engineering – and collaborate in the ongoing creation and evolution of new, constructive technologies in a complex, ever-changing world.

~  ~  ~  ~
The writer, Ira G. Liss, is assistant communications director at ATLAS Institute and a performing artist. See video of his original commentary, songs and spoken word here.

 

Exploring 3D Tactile Technologies for Visually Impaired Children

A Unique Challenge – Helping Families Create Tactile Books for Visually Impaired Children

ATLAS MS-ICTD student Abigale Stangl and computer science team work on human-centered design strategies and tools

Abigale Stangl has taken on a unique challenge. How can parents with visually impaired children create custom-tailored storybooks and learning tools for their loved ones using 3D printing technologies? How can this be done easily and affordably by parents with varying levels of computer proficiency? What kinds of guidelines and best practices will serve families dealing with these issues?

Photo at left: ATLAS MS-ICTD student Abigale Stangl and a computer science team led by assistant professor Tom Yeh work on human-centered computer (HCC) solutions to help parents of the visually impaired. She sits in the lab where computer science graduate student Jeeeun Kim fabricated the tactile book prototypes shown below.

CU’s ATLAS Institute ICTD Practicum

Stangl is conducting this research as part of her practicum semester in the last portion of her academic program at CU’s ATLAS Institute Master of Science in Information and Communication Technology for Development (MS-ICTD). During the practicum, ICTD students turn classroom theory into on-site, real-world practice.  

They team up with private or public organizations, companies and NGOs to work on solutions to various quality-of-life issues in communities located around the world.  Stangl teamed up with Professor Tom Yeh and fellow graduate student Jeeeun Kim in the Sikuli Lab of CU’s Department of Computer Science.

A Surprising Path

She came to this project from a surprising path: environmental design and landscape architecture. Quite a leap, one might say. Perhaps.

However, the challenges she took on in her previous field called on her to design and problem-solve across multiple disciplines – urban design, horticulture, community relations, etc. Her proven ability to work comfortably with a diverse set of specialties continues to serve her today in the field of ICTD.

The People Part

There are multiple dimensions to this project. One of them is people, of course – parents, children, teachers, developmental and behavioral psychologists and computer scientists.

“As a cultural component of the work,” she explained, “I regularly visit Denver’s Anchor Center (a preschool for visually impaired children) to observe the ways small children play and learn. They range in age from several months old to five years old. I learn a lot from seeing how teachers and parents interact with the children.”

From this steady observation, Stangl gains insight into what sorts of learning tools might best serve children and parents. One of her insights appears to be a universal truth about learning.

Stories are Fundamental

“We learn from stories and story telling. Stories are fundamental. Just as parents of children (with fully functioning senses including sight) enjoy the intimacy and connection of reading to their children with picture books and storybooks, parents of visually impaired children have the same desire to participate in their children’s lives.”

2D to 3D Software Interface

Screen layout at left conceptualizes what a 2D to 3D software package could look like. As its subject matter, Stengl features the classic children’s book “Good Night Moon.” The goal is for parents and teachers to be able to create 3D objects from existing 2D children's books that visually impaired children will then be able to explore, play with and learn from.

Stangl continued, “For the visually impaired, tactile development is vital. Before children can learn to read (with Braille), their sense of touch needs to be practiced,  strengthened and heightened. This comes through the exercise and exploration that tactile books provide. Of course, it’s not only about touch. It’s about the understanding and ‘seeing’ of a greater world with all its complexities and context that is made available through the window of touch.”

The Technology Part

As mentioned, the technology portion of the work is being developed by a team of colleagues in the Sikuli Lab of CU’s Department of Computer Science. Graduate students collaborating in the work include Jeeeun Kim who contributes to the research by experimenting with software, hardware and a variety of materials to fabricate tactile prototypes. (See below.)

Photo at right: Five textured layouts were  fabricated by computer science graduate student Jeeeun Kim. Each one could potentially be used as materials for tactile storybooks. Clockwise from left: Plastic yellow layout shows a tactile illustration of a room interior, formed by a 3D printer; three executions of a diagram of an egg uses three materials: 1. Cut and folded blue paper; 2. Multi-layered laser-cut wood; 3. Etched red plastic; sitting to the right of a pen, a transparent plastic sheet becomes a base for raised forms that illustrate a room interior, made from a glue gun.

Several faculty members serve as principal investigators and senior advisors. All contribute to moving the research forward. One of Stangl’s computer science professors, Tom Yeh, proposed the project and ignited her curiosity about tactile perception and interface design for children with visual impairments.

User-Centered Design

“I took a class with Dr. Yeh in human computer interaction (HCI) and human centered computing. What I learned in class beautifully fit my worldview and my goals – make sure projects are user-centered. The work must begin and end with the needs of the user," Stengl continued.

Photo at left: At $2,000, this MakerBot 3D printer represents a price breakthrough that could make it possible for parents of visually impaired children to print their own tactile learning aids at home.

“Given my design background, I’ve seen how both environmental and social information must be synthesized into solutions that are sensitive to the needs of whoever you’re designing for – in this case – teachers, parents and their children.”

~  ~  ~  ~

Abigale Stangl and six other ATLAS MS-ICTD students presented the work they completed during their practicum semester on April 10th. Videos of their presentations will be available online.

~  ~  ~  ~

Link to ATLAS Institute’s MS-ICTD program:

http://www.colorado.edu/atlas/newatlas/masters/

~  ~  ~  ~

The writer of this article, Ira Liss, is ATLAS Institute's assistant director of communications and also a pianist, singer/songwriter and performing artist. See videos of his original work. Contact him here.

Application Developers Alliance and the University of Colorado Boulder’s ATLAS Institute Team Up To Advance App Innovation Research

Press Release
Washington, D.C. (May 1, 2013) - The Application Developers Alliance has established the Alliance Research Fellowship at the University of Colorado Boulder’s ATLAS Institute. The goal of the fellowship is to further understanding of the dynamics at play in the app development ecosystem. The ATLAS Institute has appointed Ph.D. student Sid Saleh the inaugural Application Developers Alliance Research Fellow.

“The Alliance is thrilled to partner with the ATLAS Institute to provide first-of-its kind insight into the developer community and innovation in the apps industry,” said Jon Potter, President of the Application Developers Alliance. “This allows us to tailor our programs and advocacy efforts to meet the needs of developers. Furthermore, our ground-breaking research will give policymakers insight into the work developers are doing, their contributions to our economy, and the challenges they face.”

Saleh has launched two research inquiries. The first is an ongoing poll of application developers, aiming to accurately describe the developer community. The survey includes the number and different types of developers, the kind of work they are doing, the specific challenges they face, and how these are evolving over time. The second study was launched in conjunction with the Alliance’s Developer Patent Summits and looks into developers’ experiences with the software patent system.

“We are honored to have the Application Developers Alliance associated with the work of ATLAS through support of our Technology, Media and Society Ph.D. program,” said John Bennett, Director of the ATLAS Institute and Archuleta Professor of Computer Science at the University of Colorado Boulder. “The ATLAS Institute’s focus on interdisciplinary, technology-related research makes us particularly qualified to partner with the Alliance in exploring a wide variety of issues that matter to the developer community. Our scholarship incorporates real-world data and practical challenges. We expect the research the Alliance Fellow conducts will provide unique insight into an industry that has such an impact on global digital society.”

About the Application Developers Alliance
The Application Developers Alliance is an industry association dedicated to meeting the unique needs of application developers as creators, innovators, and entrepreneurs. Alliance members include more than 20,000 individual application developers and more than 100 companies, investors, and stakeholders in the apps ecosystem.

About the ATLAS Institute, University of Colorado Boulder A campus-wide initiative, the ATLAS Institute leads discovery and innovation at the intersection of technology and society. We seek to better understand the interaction of people with information and communication technology (ICT), and to realize the full potential of that interaction. ATLAS interdisciplinary programs help develop creative designers, critical thinkers, effective leaders, capable learners, transdisciplinary innovators and engaged global citizens. Contact: Courtney Lamie, Application Developers Alliance 202-250-3006 (desk) Bruce Henderson, ATLAS Institute 303-735-0899 (desk)

International multimedia artist Nicholas Jaar speaks at ATLAS at 6 p.m. this Friday, April 26, as part of two free ATLAS events

1. Nicholas Jaar – The ATLAS Speaker Series and the Communikey Festival of Electronic Arts present Nicholas Jaar, an international multimedia performing artist, who will talk about his digital music work and creative process.
6 p.m. this Friday, April 26, ATLAS 100, Cofrin Auditorium, first floor of the Roser ATLAS Building

2. Masaki Batoh – The ATLAS Center for Media, Arts and Performance and the Communikey Festival of Electronic Arts present Masaki Batoh and his performance of Brain Pulse Music, which features music based on brain waves.
7:45 p.m. this Friday, April 26, ATLAS Black Box theater, downstairs, lowest basement level B2, Roser ATLAS Building

Nicholas Jaar is founder of the record label Clown & Sunset and released his debut album "Space is Only Noise" in 2011. Record label colleagues plus emerging Colorado-based music producers will join him for a Q&A discussion.
Jaar has performed around the world, including an acclaimed five-hour improvisation at NYC’s Museum of Modern Art that he will perform again during the Communikey Festival.

Masaki Batoh's Brain Pulse Music is the result of research into the bioelectric functions of the human brain combined with the traumatic aftermath of Japan’s Great East Earthquake. When first conceived, it was to realize music from extracted brain waves.

Both events are free and open to the public.
Audiences are advised to arrive 10 minutes early.
Seating is limited and first-come, first-served.

The ATLAS Speaker Series is made possible by a generous donation by Idit Harel Caperton and Anat Harel.

Live and in Concert: New Sound Controller Technologies Enhance and Create Music

Creating and Controlling Sound
Following its mission of applying technology to a wide range of disciplines - including the performing arts - CU's ATLAS Institute recently presented performances that explored the use of controller technologies to produce sound and music.

On March 2, 2013, computer science Ph.D. student Charles Dietrich collaborated with the Boulder Laptop Orchestra (BLOrk), the ensemble-in-residence of the ATLAS Center for Media, Arts and Performance, in the creation of a piece called “Gestures.” In this piece, three music students moved hands and fingers in front of 3D cameras that were able to recognize the three-dimensional positions of the center of the hand plus fingers.

Three students control sound through the movement of their hands which are tracked by 3-D Intel cameras. Images projected on the screen behind them show how the camera's software is able to diagram the movement of their hands by tracking the center of their palms and positions of their fingers.

A Gesture-Based Music Controller
"Our hands are so expressive," Dietrich said. "A technology that can track and record their subtle movements can also help us develop new applications – in music, sculpture, visual arts and CAD programs. Those who study the body language of various cultures may have new tools for research as well."

The hands of doctoral student Charles Dietrich are read by an Intel 3D camera mounted to the top edge of his laptop. Inset image shows how his hands are interpreted by the software. Brightly colored lines are superimposed over his palm and fingers and change angle and shape as he moves. In this way, the software can detect whether hands are open or closed or shaped in a way that might trigger a change in the sound specified by the programmer/developer/musician.

The camera incorporated a depth camera and a standard color camera. The depth camera used infrared light and an infrared detector to determine depth by time-of-flight (ToF) of the light, similar to radar, but with light. The depth camera and color camera data were processed through Intel software to determine the hand landmarks.

Dietrich used data from the Intel-developed camera software to assign sounds to hand movements. For example, the altitude of the left hand determined note or pitch. Opening or closing the right hand determined whether a note was played. The altitude of the right hand controlled loudness. In this way, Dietrich’s gesture-based controller functioned as a musical instrument.

13 Pedals, No Hands
Another controller used at the same concert was developed and played by College of Music Ph.D. student Hunter Ewen. Dubbed the “SCREAMboard” (an acronym of “soloist-controlled, real-time, electronic audio-manipulation board”), the foot-operated musical interface was used for real-time composition and improvisation.

Doctoral student Hunter Ewen presents his foot-controlled controller that allows hands-free control of 13 sound parameters. “My goal is to have the controller allow electro-acoustic musicians a hands-free way to control all the special effects and sound features they could want. Achieving this with earlier tools has been a difficult, awkward and often expensive process,” Ewin said.
Thirteen pedals controlled recording, playing, looping (after being recorded, a sound or sound sequence is played back repeatedly as a “loop”), synchronizing loop lengths (managing the way that multiple loops fit together or align with one another), panning (controlling the way sound is thrown around a room given its arrangement of speakers) and volume.

In one piece, Ewen (shown at left and below at right) used his voice, a kazoo and several unconventional instruments to record and loop a variety of sounds that he layered into complex textures.
The colorful diagram of vertical bars projected on the screen behind the performers shows the status of the 13 different controller pedals.

In another piece, John Gunther (shown at left), associate professor of jazz studies and co-director of BLOrk, performed a solo piece using Ewen’s interface. He played his jazz flute, recording and looping melodic sequences that harmonized with one another to become a rich background upon which he improvised a solo.

From One Mouse to Many
John Drumheller, a CU College of Music instructor, director of music technology and BLOrk co-director, talked about controllers. “In the past we were limited to one slider or one controller when we used a computer and a mouse. We could control only one aspect of sound with that mouse. But now, with the introduction of the tablet, smart phones and touch screens, each finger can control a completely different sound or quality of sound.”

BLOrk co-director John Drumheller performs on a tablet during a December 2012 concert in the ATLAS Black Box theater.

It is now possible for our 10 fingers to control 10 sounds or aspects of sound (pitch, volume, rhythms, harmonies, etc.) simultaneously. Of course for centuries, the piano has allowed our 10 fingers to make 10 different sounds at the same time, but we are talking about the ability to assign a different sound, instrument, rhythm or dynamic to each finger. (This is similar to Hunter Ewen allowing us to assign and control 13 different sounds or parameters using his foot pedal control device.)

Four Controllers, One Touch Screen
In the music lab, Drumheller showed me his tablet for a moment. He set its screen up with four rectangles of different colors. As I touched the screen, I let my fingers slide around the surface, tap and wiggle. Instantly (much to my delight!), four different exuberant sounds were generated simultaneously as my fingers moved.

Soon, as Dietrich’s gesture-based technology progresses, we will be able to move hands and fingers in the air around us, unconfined by screen, tablet or keyboard. Our movements will instantly become sound. Dance and music, always complementary art forms, may become one.

~ ~ ~ ~
See additional photos of the March 2nd concert of the Boulder
Laptop Orchestra (BLOrk) that took place in the downstairs
ATLAS Black Box theater.

Get yourself on the ATLAS email list. ATLAS concerts, dance/theater performances, art installations and Speaker Series talks are always free and open to the public (well, with very few, rare exceptions).

Look into a list of upcoming ATLAS events.
Perhaps there are several you'd like to attend.

Contact the writer: Ira Liss is ATLAS Institute's assistant director
of communications and also a pianist, singer/songwriter and performing artist. Enjoy videos of his original work.

The ATLAS Speaker Series, which is made possible by a generous donation by Idit Harel Caperton and Anat Harel, hosts distinguished visitors from academia, industry and the arts as part of the ATLAS Institute’s mission to explore information and communication technologies and their effect on society.
The series is an educational and experiential resource for students, faculty and the larger community to discuss the challenges, opportunities and innovative applications of technology. Talks usually run from 4-5 p.m. in the Cofrin Auditorium, ground floor, ATLAS 100 (enter from lobby), unless otherwise noted.
The following speakers have been scheduled for the spring 2013 semester:

Digital Media and Music as an Instrument for Social Change

Paul Miller (aka DJ Spooky) will discuss his recent work exploring digital media, music and the ways that art can open minds and help people gain new perspectives on issues like climate change. He will perform his multimedia electronic compositions (using his own DJ app and wall to wall graphics) accompanied by CU College of Music cellist Megan Knapp and violinist Emily Lenck. Miller is a multimedia digital artist, musician, composer, remixer, author of “The Book of Ice” and the first artist-in-residence at NYC’s Metropolitan Museum of Art. His visit is a collaboration between ATLAS Institute, the Program for Writing and Rhetoric and the President's Fund for the Humanities. 6-7:30 p.m. Wednesday, Feb. 6, downstairs Black Box theater,
lowest basement level, B2

How Open Source Software is Changing Technology

Stormy Peters will look at how we can predict the future of technology by observing the use and development of such software. As examples, she’ll show how nonprofits and open source software projects set technology directions and make bold statements about how to make the world a better place. She will also discuss her work at Mozilla and share insights about the future of information technology (IT).

Stormy Peters is director of websites & developer engagement at Mozilla. She is an advocate and supporter of open source software and its potential to change the software industry. She is also the founder and vice president of Kids on Computers, a nonprofit organization that sets up computer labs in developing countries. 4-5 p.m. Monday, Feb. 18, Cofrin Auditorium, ATLAS 100

Women and the Web: Bridging the Gender Internet Gap in Developing Countries

Renee Wittemyer will discuss the data on the large Internet gender gap in developing countries and the social and economic benefits of securing Internet access for women. Recent studies find that on average – across the developing world – nearly 25 percent fewer women than men have access to the Internet and the gap soars to nearly 45 percent in sub-Saharan African regions. Wittemyer will discuss this research and a call-to-action to double the number of women and girls online in developing countries from 600 million today to 1.2 billion in 3 years.

Wittemyer, director of social impact in Intel Corporation’s Corporate Responsibility Office, develops strategies for Intel’s girls and women’s campaign and manages relationships with strategic alliances, such as USAID, NGOs, and U.N. Women.
4-5 p.m. Monday, Feb. 25, Cofrin Auditorium, ATLAS 100

Education at the Intersection of Computer Science and Music

Ge Wang will discuss the transformative possibilities of music and computing to make art, strange new instruments and connections to people around the world. His talk will explore laptop and mobile phone orchestras, computer music languages, social music apps like Ocarina and Magic Piano – all examples of an emerging, growing space where computers, music and people interact. Wang is a Stanford University assistant professor in the Center for Computer Research in Music and Acoustics (CCRMA) and researches programming languages and interactive software systems for computer-generated music. 4-5 p.m. Monday, March 18, downstairs Black Box theater, lowest basement level, B2

Learning to Love Technology by Making Arts and Crafts

Leah Buechley will discuss her work developing accessible technologies that allow people of all ages – including those who might not otherwise be attracted to computing – to sketch, design, paint and sew while incorporating electronic technology. As examples, she will show student projects that blend computing with traditional arts, crafts, textiles, paper and wood. A developer of the LilyPad Arduino toolkit, Buechley received her master’s and Ph.D. degrees in computer science from CU. She is an associate professor at MIT and directs the High-Low Tech Media Lab. 4:15-5:15 p.m. Wednesday, April 10, Cofrin Auditorium, ATLAS 100

Mystery, Music and Digital Innovation

Nicolas Jaar, an international multi-media performing artist, will talk about his digital music work and creative process. Record label colleagues plus emerging Colorado-based music producers will join him in a Q&A discussion. Jaar is founder of the record label Clown & Sunset and released his debut album “Space Is Only Noise” in 2011. He has performed around the world including an acclaimed five-hour improvisation at NYC’s Museum of Modern Art. The talk is a collaboration between ATLAS Institute and the Communikey Festival of Electronic Arts, http://communikey.us. 6-7:30 p.m. Friday, April 26, Cofrin Auditorium, ATLAS 100

Upcoming performance at ATLAS: Tame Your Man

"Tame Your Man is my first work dealing directly with my sexuality as a gay composer and my experience of masculinity within gay culture," explained director and composer Nathan Hall. "Over the course of the half-hour piece, a pianist will be bound to his instrument and his abilities to use the full range of the keyboard will become increasingly restricted. The performance will explore surrender, pleasure and trust in an abstract, other worldly atmosphere."
7:30 p.m. Friday & Saturday, Nov. 9 & 10, Black Box theater