Note: the following overview is a preliminary sketch. Comments are welcome. -Ken

Center for Responsive Media Technologies

  • Mission
  • The Center's mission will be to develop research, education and industry programs to create Responsive Media Technologies. Our goal is to enhance human communication through the development of agents and interfaces that are sensitive to, and respond effectively to, human affect.
  • Research/Education/Technology Transfer
    Our research focus will be on the creation of the underlying technologies that enable such agents and interfaces. Our education focus will tie together our existing academic disciplines, as well as to create new ones, that prepare students to create the new industries that will form as a result of Responsive Media technology. Our industry focus will contain institutional mechanisms for the commercialization of technology developed from the Center's research, as well as opportunities for students to work with participating corporations and to create new companies around the technologies they help to develop.

  • Research
  • Our research focus will be to bring together the various software engineering, research, and artistic interface tools so that we can create viable communication agents that understand and synthesize speech, are reponsive to human gesture, voice and facial expression, can make decisions about how best to respond to humans, and can be convincing and effective in simulating voice, gesture and body language.
  • Tools development
    Much of our focus will be on tools development. In many of the component areas that fall under our mission, there is currently a gap between the tools available to programmers and those accessible to domain experts. Much of the effectiveness of analysis and synthesis of speech, gesture, facial affect, and other affective channels will be dependent on the creation of tools that can be manipulated easily and properly tuned by skilled domain experts in these areas.
  • Canonical applications
    The Center will identify canonical applications that can bring together key research threads, as well as being able to directly impact Education and Technology Transfer. One canonical application would be an educational tutor that would engage a student on a deep level, would be sensitive to when the student is focused, lost, confused, interested, ready to move along faster, or in need of diversion or greater challenge, and can present a convincing and appropriate persona while making these decisions. To effect this, we would also need to research information filtering and retrieval engines and automated multimedia authoring and presentation tools.

  • research areas:

  • Education
  • Our education focus will work through the above research mandate to tie together a number of important academic disciplines, including computer science, interactive interface design, business and information sciences, and arts education for film, television and animation. We will add interdisciplinary courses to create an educational base that will allow a student to become expert in at least one of these areas while being informed in the others, and will provide substantial opportunities for students in these related disciplines to work together in joint projects.
  • Cooperation among existing programs
    NYU is well known for its graduate programs in fields related to this research. The Interactive Telecommunications Program in the Tisch School of the Arts is one of the leading programs in the country for the study of multimedia technologies and their uses. The Computer Science Department in the Courant Institute of Mathematical Sciences, has strong Masters and PhD programs in related fields such as computer graphics, multimedia, and natural language processing. In addition, the Stern School of Business has graduate programs in technology transfer, information technology, and entrepreneurship. By providing a physical location, interdisciplinary research projects, and collaboration with industry, the Center will become a place for graduate students in these various programs to cooperate and further their education.
  • New media specialty
    NYU is also in the process of creating an undergraduate specialty in multimedia that will be available to students from many of the schools in the university. Required overview classes will be followed by specific classes within each student's major (computer science, film/tv, education, business, etc). Included in the program will be one or more project classes that bring together students from the various schools in a small-business setting where they will design, create and market a multimedia product. The Center will become the ideal place for these collaborative efforts to take place. The more successful projects will also have the opportunity to get the attention of the Center's industrial partners.

  • Technology Transfer
  • Procedures will be instituted that enable the rapid licensing of technology and formation of companies by students, faculty and staff based on research developed by the Center.
  • Industrial collaboration
    A key component of the Center's mission is its Industry Collaboration Program. Industry participation ensures that there are institutional mechanisms for the commercialization of technology developed from the Center's research. Our industrial transfer/impact focus will create proactive opportunities for students to work with participating corporations, as well as to form themselves into complementary teams that can form new small companies around the use of the technologies that will emerge from the research focus.
  • Corporate partners
    The Industry Program at the Center will be anchored by a core group of large corporations interested in Responsive Media research. These companies will not only provide additional funding for the Center, but will be working closely with the Center's management to shape the strategic direction of the Center's work. Student/researcher exchange programs between the University and the corporations' R&D departments will encourage technology transfer and enhance the University's effort to graduate qualified technical people.
  • Technology Incubator
    Another key component of the industry program will be a technology incubator, which will offer space, equipment, and administrative support for start-up companies based on Responsive Media Technology research. The Center will act as a catalyst for student teams from varied discplines and schools within the University that will work togethor, first as part of course work, and later, as a commercial venture. Entrepreneurs from outside the University will also participate in the Incubator and can draw talent from the University's interdisciplinary Multimedia program.
  • Small Business Development
    The Center would also continue the current work of the NYU CAT Small Business Development Program to ensure that technology developed at the Center would be used by small- and medium-sized companies. The Responsive Media Technology Research Center would act as a bridge between its large corporate sponsors and the budding entrepreneur new media companies. The resulting partnerships would be an entrepreneurial enterprise using cutting edge technology and backed by large corporate funding. There are a plethora of candidates for small business partnerships in New York City in particular.