I Have a Dream – the Online Semantic Collaborative Design Space
March 8, 2010
Note that the title of this post has all the keywords I need to make sure Google ranks it above others. If you’re reading this in 2020, I want you to know that we had to roll over and do stupid tricks like that to please search engines and their bots, because the semantic web didn’t exist back then and there was a search engine called Google that thought looking at keywords in the titles of HTML files was a good way to find information.
I Have a Dream – the Online Semantic Collaborative Design Space
In 2002, the Lord of the Rings trilogy impressed audiences with realistic epic-scale fight scenes featuring thousands of virtual actors – orcs, trees (Ents), goblins, monster elephants, etc. To make the big army scenes, Sony Digital worked with Weta Digital in New Zealand to make each virtual actor its own autonomous program. The “actors” could interact with the virtual scenery, step over logs or dead soldiers, fight or run depending on the situation, and recognize friends from enemies. The actors were able to make their own decisions and have their own agendas.
In 1990, Boeing and Dassault Systemes created a virtual workspace for designing the 777 aircraft. Over 10,000 contributors used the same virtual space to design a virtual airplane using virtual parts. But the result was very real, and the 777 has been a huge success. They continue to use the same system to make modifications for production requirements as planes roll off the assembly line. Today, the next generation of that system helps them build and test the 787 Dreamliner airplane, which will be delivered later in 2010. Each part is created and installed in the virtual plane, which they can test for vibration, heat, air flow, structural integrity, weight balance, and flight characteristics. In fact, they built the simulator and pilots had a chance to fly the virtual airplane, allowing the designers to make changes years before building the first plane. The 787 is considered the world’s largest current industrial project (from an information standpoint), and it can only be done in a single online virtual space, rather than sending documents all over the place.
In chapter 9 of Pull, I wrote a scenario about a future home, which is designed, approved, constructed, marketed, and operated all in the same virtual space online. The information for the home never moves. People come in and use tools and rules to do what they need to do, and the virtual home controls what happens in the real home every step of the way.
I’ve been dreaming about this for ten years. I believe I’m in the right place at the right time to bring this shift to all design disciplines:
- Interior design
- Transportation design
- Product design
- Industrial design
- Film production
- Graphic design
- Exhibit design
This approach is vastly different from what we do today, which is little more than an electronic version of our 19th century drafting methods. Today, we have desktop applications that mimic our drafting tables and drawing tools. We send documents back and forth. We use “the cloud” to hold documents. That’s not collaboration; that’s storage. We’re still working alone on our big applications, and the applications require lots of resources and maintenance to stay working. We spend a lot of time dealing with application issues, system issues, back-ups, version problems, and re-doing work others have done or that has been lost. Putting applications on the cloud doesn’t make these issues go away entirely, but making applications much smaller will help us scale up our software efforts.
We now have what we need to do things right. We have broadband connections and a scalable cloud-based infrastructure for running applications online. What we’re missing are the standards and the solutions to bring these environments to life. My goal is to make a new online platform for designers, one that replaces the “paper” all applications use with an online workspace that has everything needed in one place. In my dream:
- You start with the requirements and build everything in a single place, so the early sketches become the final finished product and the data never ever moves.
- All contractors and participants can create virtual elements that plug in to create “what if”?, competitions, or multiple-bid situations.
- The final design becomes a production environment, driving what happens in the real world.
- The final data is used for distribution.
- The final data can be used for marketing and sales.
- The final data is then transfered to the new owner.
- The space then transitions to ownership, operation, and maintenance (Product Lifetime Management).
- All the rules and constraints will be semantic.
- All documents, constraints, and requirements apply semantically.
- Actors, parts, assemblies, elements, and structures snap into place and recognize their environments.
- Applications come in as needed. Applications typically do small jobs and are assembled ad-hoc to meet the current needs of the project. The age of huge applications comes to an end.
- Employees and contractors come to the space as needed. They are given permission to do their work and their permissions are managed.
- All stakeholders can see the project and participate in decisionmaking as needed.
- You can run the project forward to see how it looks or performs.
- You can run the project in reverse or back to the beginning to review any elements or pursue a new avenue of design.
- You can mine the project to retrieve records or answer questions.
- The data formats are open and royalty-free, governed by nonprofits, not corporations.
- As much as possible is on the web. As little as possible is stored on local machines or in deep databases. The goal is that all of this runs inside a (future) browser.
- The project should hook to all other related systems: vendors, suppliers, contractors, regulators, government, lenders, users, and their guests.
- Markets should make elements, parts, and apps available for rent and purchase.
Once we have a few standards in place, we’ll have to promote new markets to help move things forward. A few scenarios will help:
In film production today, all the computer networks are private and use proprietary data formats to move files from one computer to another. A few large studios are starting to create collaboration spaces, but they are not scalable to the entire industry. To create a scene using the future open system, you’ll start with a blank 3-d workspace online, where everyone can collaborate. Then you’ll add the elements you want – mountains, meadows, rivers, weather, cityscapes, streets, buildings, rooms, hallways, doors, windows, furniture, appliances, artwork, lights, plants, animals, insects, people, etc. To do that, you’ll go shopping for virtual actors and props online. Because of the standards, they will be like clip art or stock photos. If you want a talking penguin, just specify what kind of talking penguin you are looking for (Chinstrap? Emperor? Rock Hopper?). Thanks to the standards we’ll create, these virtual actors will come from all around the web for auditions. Once you choose the one you want, he (if it’s a male) will fit right into your online environment. He’ll be interoperable, so he already knows how to interact with the floor, steps, furniture, doors, windows, switches, lights, etc. Actors will automatically know how to interact with each other. Using common standards, everything will be compatible and interoperable, so you can search for everything you need online. You’ll move the actors through the scene, give them their lines, and run everything together to see how it looks. Need another monster? Go get the latest one and throw him in – he won’t bump into anyone unless that’s the kind of monster he is. If he throws a chair toward a window hard enough, the window will break. Change a donkey to a zebra in seconds, but watch out – a zebra will eat the grass down shorter than a donkey does, and the grass will take longer to grow back. All these behaviors are already built in. If you turn off gravity, all the actors will float around and interact with whatever is in their path. None of this is pre-scripted; the actors are “smart” enough to know how to recognize their whole environment. It still requires people with great skill to make a compelling scene, but most of the mechanics of just getting a dog in a room and having her bark on cue will already be done. Animators won’t have to constantly reinvent the wheel or write software to practice their craft. Markets will be much more efficient than today’s stock photo markets – the data will be linked to a central repository and findable using semantic search. So you’ll be able to find a virtual border collie that can do backflips, and if you can’t find it, it probably doesn’t exist.
In graphic design, we practically have to start from scratch. Designers are used to using huge programs like the Adobe CS4 suite, but those products are the pinnacle of electronic tools based on their 19th century counterparts. The biggest qualitative innovations in design using computers have been a) the revision possibilites and b) that you can start with a sketch and go all the way through to the finished design using the same environment. But that’s not how we’re going to do it in the future. In the future, all design documents will have structure. If you’re designing a poster, that’s different from a business card. An invitation is different from an annual report. Again, designers will be able to plug and play various elements and they will automatically understand their environment. There will be much more assembly and modification of pre-built parts than starting from scratch each time. All the resources will be live, online. Collaborators will come in, do their jobs, and leave the work ready for the next step. Many documents are data driven, and the data will not just be live, it will be semantic, so it will be much easier to work with and the environment will be able to deal with it in a smarter way, without extra help from humans. Of course, the semantic web will change the nature of many kinds of documents; for example, most documents disigned to gather and display data, for example, will simply go away.
In product design, we need to integrate interoperability, virtual testability, prototyping, virtual manufacturing, and supply chain. It’s a long way from conceptual sketch to a product’s birth certificate going into an owner’s personal data locker, but we’ll get there eventually. This involves building the product birth certificate infrastructure, but that’s likely to emerge anyway. I’m in touch with at least two groups working in that direction. So we can start by creating real collaboration spaces where designers can try things. Dassault has built much of this software, but it’s not an open platform. That’s what we need, in addition to open formats so everything can plug and play. If there are constraints, they should be built into the environment so that the product must comply at all times. In other words, using this system with constraints turned on, it should be impossible to create a product that doesn’t comply.
For an example in architecture, see chapter 9 of my book, Pull. The exciting thing here is that the BIM standard is really gaining momentum, so we have a platform standard coming along nicely already.
You may wonder: “How is this going to work? Standards don’t come first, they come after an innovator has established the market.” But is that really true? Look at the web itself. Look at bar codes. Look at XBRL (see chapter 5 of Pull). There are many examples where simple standards helped get things started and then the market accelerated. Without a common infrastructure, design and production will remain fragmented. We already have the precursor for many of these things, so we can imagine a smooth transition to this new world, rather than a disruptive event. Similar to the shift from analog to digital, there won’t be a single year that is the year when all design platforms went semantic. Rather, there will be a series of pilot projects, browser extensions, early-adopter tools, cloud-based apps, and successful results that lead to the next step, and the next, and the next. Salesforce.com is a great example – this is an entire contact-management and CRM platform built entirely on the cloud, and now venture capitalists are helping fund companies to continue to build out the ecosystem.
It’s a big vision, one that will need a lot of people to buy in. That’s my job. Getting started isn’t that difficult or expensive. I need to make a series of vision videos and spend a year getting out to conferences, consortia, and companies to plant the seeds.
My road map to this future is:
- Make a series of short videos showing the vision.
- Put them online and take them on the road to conferences and show people what is possible.
- Build a web site where people can collaborate to help move things forward.
- Meet with people in companies, make presentations, get private sector cooperation.
- Have a series of events in different cities to educate people and get critical mass.
- Form the groups and standards bodies necessary.
- Get governments committed to supporting the projects.
- Design pilot projects in each sector.
- Put down early versions of the platforms so people can play with them (similar to NEPOMUK and Dassault)
- Raise a venture fund to help build companies to build the tools we will need.
- Hold conferences and seminars for investors.
- Leverage the web, rich media, and the collaborative tools we already have today.
- Evangelize, evangelize, evangelize. Show results and investment payback.
- Keep the conferences going and be sure to be part of existing conferences.
There will be a long transitional period. We’ll need to help people use the tools they have today to contribute to online data spaces, and then help them migrate away from the desktop and onto the browser. Boeing and Dassault have done it. Salesforce.com has done it. There are other examples. The roadmap will help people understand where we are and where we’re heading.
Personally, I hope to play a small catalyzing role by architecting this process, getting the critical mass of support to keep it moving, and helping start some of the key companies in this new world. To do that, I want to make a set of compelling videos showing scenarios for the following design disciplines:
- Transportation design
- Product design
- Film production
- Graphic design
Each of these five videos will be around 5-7 minutes long. I will contribute my video on the online semantic data locker, which I am funding personally and should be finished by May this year. I will also need to write a white paper and build a web site explaining the project. I’ll need to build the business case, which shouldn’t be too difficult given how much money we waste today doing things the old way. Making the timeline real and showing the transition path will be key to the project’s success.
National governments and inter-government bodies will help, because it fosters growth and efficiency in their economies. I expect many consortia to play important roles as well. I would like to get at least these groups involved:
- The EU is far ahead of everyone else. They have money set aside, several projects already funded, academics working with businesspeople, and have large consortia interested. See the Future Internet project and Nepomuk.
- Singapore plans to be a major force in digital media. I hope they will get interested in the semantic side of design, as that’s where the future innovation will be.
- China should be interested. I know people in the design community there and will try to find who might support it.
- MITI in Japan should be able to help, and companies like Sony, Mitsubishi, and others will be interested.
- Korea‘s government and private sector should be very interested.
- The US government should be interested, but unfortunately it’s going to lag behind. Congress has allocated money to projects like this in the past, but it takes a lot of political momentum to make it happen. I’ll have to go to California and find companies that want to help make this world a reality. I would start with companies like IBM, Cisco, Adobe, Google, Audodesk, Boeing, IDEO, Disney, Dreamworks, and others. In addition, I hope to bring this project to the attention of the US CIO, Vivek Kundra, who may be able to help get things started.
- Canada will probably be very interested. I have to find the right people there.
Here are a few projects that help point the way:
EU Future Internet project
EU Semantic Desktop
EU ServiceWeb Roadmaps
Dassault Systemes videos
Article on Dassault and Boeing
The BuildingSmart Alliance
IBM’s Smarter Cities initiative
For me, this is a ten-year project. My goal is to help coordinate, promote, and catalyze. I’ll work with anyone, anywhere, and I’ll participate in any projects already underway. I’ll have to find the best place for me to fit in, because I don’t have funding. I’m very open to suggestions. And you can help! Please cut and paste this post into an email and send it to anyone who might be interested, or send them to ThePowerOfPull to read about it. Please give my book to people who could help. If you know anyone at the EU, Dassault, IBM, or anywhere else who might be interested, please introduce us. I’d love to hear from anyone who wants to talk about the future of collaborative online design.
If you are at the SXSW conference this week, please tweet this URL and tell people to spread the word.