Or: How I Learned to Stopped Worrying and Love .MHX2
So I just had a major breakthrough (I’m hoping!) with my NPC interactions. I had glanced once or twice at the work Thomas@MakeHuman had been doing, but in my earlier, more sleep deprived days of DaddyDev, I never quite grasped the concepts he was working on.
Ye gods, I’m so freakin’ thrilled I went back to take a second look.
I took one of my MakeHuman characters (the subway token booth clerk) and, after installing the .mhx2 files in Blender and MakeHuman; I re-exported him put him into Blender. Granted, I had to grasp there was an advanced options switch on the Blender import side – but once I activated the face shapes, I was blown away.
I had visemes (phonemes) already plugged in. I could export expressions from MH. Once I added the MakeWalk addon, I could import Mo-Cap .bvh files and apply them to my rig. Delving into the online manual (and a profound thanks to the dev who still does written manuals, I’m sure I will rant yet again about how much I hate video tutorials) I saw there were even more options to smooth out the mo-cap motions, edit the location / rotation of bones and once I figure it out, append other .bvh files. What really blew me away was the ability to import MOHO (.dat) lipsync files from Papagayo. And I loaded visemes ON TOP of the already loaded .bvh!
With the ability to make a character walk, talk and emote, suddenly a whole new facet of story telling opened up for me. Since I’m not big into programming, I’ve been using Playmaker to handle some of the basic interactivity; and after seeing the dev who made the wonderful Rift demo, ‘Coffee Without Words‘ made an asset that mimics human eye movement – my NPCs came alive.
Be forewarned, my demo video is NSFW; I made a MH woman with some extended clothing assets, I didn’t realize the bathrobe was translucent, so you see some breasts.
Lastly, after having tinkered with this for a bit, I got an email saying that Adobe had released Fuse, a character creation that tied in with Mixamo and the Mechanim animation system Unity now favors. I tried Fuse, made a character, got a walk animation added and plopped it in Unity (and Blender too, just to see what it would do, if I needed to tweak it)
The character worked, it did its walk cycle and looked ok after I changed its skin shader from transparent to opaque. I did notice that it can be lip-sync’ed, but only with a $35 additional purchase, and a new piece of software to learn and try to integrate. When I pulled the Fuse character into Blender, it was huge, distorted and had no face-rig that I could discern.
So Fuse wasn’t all that tempting, considering I’d have to shell out $$ for SALSA, and I suspect Adobe will be quickly adding a subscription fee like all its other products, I’d like to keep my dev budget as free and OSS as possible. Perhaps I’ll just create and export a bunch of random characters to populate my subway scene with, since MakeHuman doesn’t have the widest range of clothing options, the Fuse NPC’s will shake things up a bit visually.
Now I have to sit and watch about 2,000 .bvh animations to find what actions I think will go well with my actors…
So, the wife was kind enough to give me a daddy-cation last month, which I took to go see Steely Dan in NYC.
I also took my nice DSLR to go roam the city streets and collect texture maps, a favorite pastime when I lived there. I’ve collected terabytes of photos that have yet to be tiled, tone mapped and color corrected. Yet an other favorite pastime of mine.
Another thing I did was to try and get decent shots for a photogrammetry idea that I had: create a NYC garbage can. I had been reading on the /r/3dscanning subreddit about it and had tried a year or two back with VisualSFM, with mixed results. I recently read about Autodesk’s Memento and decided I’d give that a try.
I was absolutely shocked when I added the resulting .FBX file to a Unity scene and the sheer level of detail on the street, let alone the sidewalk. the can itself was problematic because it had intricate detail and ended up with several holes in it, I suppose I could fix those in Blender, but would have problems getting the textures realigned.
I’ve seen others recreating rooms in VR using this method, and one of the best ones, IMHO, is the 4th Floor Studio Apartment – while not a game level environ per se, a place I loved putting on my goggles and visiting anyway.
So, if you read any dev blogs, they’re always chock full of advice, ranging from the latest tips and tricks on every piece of software available, to breakthroughs and milestones reached; but inevitably there are some rules that people put forth as sacrosanct.
Well, I’ve decided to break a couple.
One that always comes up is: Finish what you are working on before starting something new.
The problem is; I’ve never worked that way. As a painter, I need to have multiple canvases going at once. If I stare at one painting too long, I get drawn into a funk of ‘sameness’ – its the same reason I can’t sit and binge watch a whole season of TV shows. I start over-analyzing and I get resentful.
So, I’m giving myself permission to take a break from BUM and work on an earlier project, an adaptation of the play / short film “Sovereignty” that I had the honor of working on a few years ago. By trying to improve what I’ve started with what I’ve learned working on BUM, I can tackle issues that stalled that project and when I get stuck there, I go back to my other project and the 2 of them become like ladder rungs I can climb at once. Playmaker hasn’t been the magic bullet I had hoped for breathing life into my characters and if I continue to be stuck, I might start trading art for programming…
The other rule that people harp on is: Don’t try and do to many projects at once.
This one might apply a bit more, but I can at least start laying the groundwork for something that might be a breakout piece for me. My eldest child is on the Autism spectrum and is fairly non-verbal. In every other aspect, he’s a normal kid – loves climbing, exploring and watching cartoons, has a fondness for sweets and doesn’t like bedtime. He just doesn’t talk much. One thing he truly is engrossed in (and as an artist I’m thrilled by) is drawing. He has gone through several boxes of sidewalk chalk this summer and every chance he gets, he’s outside covering my garage floor in layers upon layers of drawing. When he comes out, he’s covered head to toe in chalk dust.
But his drawings are compelling. Some are obviously cartoons he’s seen or toys or trains, but others are simply things he’s seen in his day-to-day life. And I think they’d make great sprites for a 2D scroller / platformer.
Iknow, I know – I can’t take on a 3rd project – none of them will ever be finished, but whle I’m cleaning up the yard or watching the kids play, I can snap a few photos and throw them into Photoshop to start prepping them to animate. After all, I’ll need another project as soon as I finish one of them, right?
Exactly one year ago, I was on vacation with the wife and kids when I got a text from our wonderful neighbor who keeps an eye on our cat and waters the garden while we are gone – I just got a large box delivered on our porch, perhaps a certain package I’ve been waiting for?
As an avid long term gamer (you should have seen my anticipation downloading the Quake 2 demo on dial-up!) and have always been drawn in by the interactivity. I’ve never been a fan of passive tv watching, and, after our family got an Atari 2600 one Christmas, its been a steady evolution of interactive adventures. Beating “Adventure.” The thrill of solving Infocom’s “Wishbringer,” on our family’s first computer. Being blown away by the graphics of “Another World” on the Amiga. Further blown away when I purchased my first PC, a AT&T Globalyst desktop with a CPU clock in megahertz, no math co-processor and playing Quake 1 running on a 15 inch monitor at the smallest possible resolution and still loving every second of it (I still often play the Quake 1 soundtrack as background noise when I’m working), playing the Half Life series, Thief series and many more hours than I care to claim.
And yet, I still craved more interactivity. I downloaded mods to make games anaglyph 3D. Adding surround sound systems to my gamning rigs. I wanted to mod Wii controllers to implement Johnny Chung-Lee’s head tracking. Still nothing grabbed my attention like the whispers I saw on Reddit about a VR headset that worked. I was instantly hooked and spent a lot of time learning everything about it that I could. When the DK2 was announced, I sold a couple of paintings to purchase it. I think the wife is still a little off put over that decision. And I waited for the day that my shipping notice would arrive.
Of COURSE it had to be while we were on vacation. I gave my neighbor dire warnings on how important this package was. Offers of booze, money, my firstborn child to ensure my delivery was safely in my house, under lock and key until I returned. I’m a safe and careful driver, but the return drive home I fought the urge to floor it all the way home. I found my new toy on the kitchen table and resisted the urge to skip sleeping to play with my new toy, as I had to be up in 6 hours to attend to the kids.
I managed to get a small chunk of free time the next day and hastily set up my Rift and installed the drivers and runtimes and dove right into the desk demo. And gasped. And laughed. And sat there in bewildered amazement, looking all around me – standing up, kneeling under the desk (and losing tracking in the process) and reluctantly tore myself away to attend to the real world. I was ecstatic when the wife offered to give me a night off from the kids and I eagerly dove into every demo I could download. I rode rollercoasters, visited haunted dungeons, went flying. Nothing grabbed my attention like a small demo by Brendan Coyle, he did a simple studio apartment and the moment I stepped into it –
I felt real presence. I was there. This was the small studio I wanted in NYC. I was there and it was sunny and warm out. I explored the kitchenette, the bathroom, tried to read the titles of the books on the shelf. I wanted to open the door, go down the stairs and to the nearest bodega for a bagel and cream cheese. It was that real.
It didn’t take long for me to delve into gaming on the Rift – I was surprised when Quake 2 made everyone else queasy within seconds, yet I could take hour long doses of it, even with a tendency to have vertigo, maybe years of twitch gaming had dulled my nausea reflex. I dove into Half Life 2 with a zeal – stopping to look at the decades old game and the mastery of level design: http://steamcommunity.com/sharedfiles/filedetails/?id=184922180
little details jumped out at me and again, that sense of immersion gripped me tight and I was willing held captive as long as I could afford.
Eventually, I wanted to create my own worlds – to take the ideas bubbling in the aether of my imagination and make them whole and walk through them. I picked up Blender (having a little experience in 3D work, I knew the concepts, just needed to learn a new interface) and picked up Unity – both free options to express myself; with little time and less money, FOSS options were my only choice and I try to support these and other endeavors as much as I can. I dove into making my worlds and the day came when I could cram on my Rift and hit Enter and after that all too familiar Health Safety Warning…
Ye Gods… it was a basic polygon world and primitive texture mapped, no fancy normal maps or SSAO shading – but it was mine. And I could walk through it. All the paintings I’ve done that I wished I could walk through – worlds in my head I wish I could visit, all of these were in my grasp, just yearning for my feeble skills to flesh them out and make them tangible, more cogent…
As I played – as I built, as I worked my way through trial and fumble – other things tickled the back of my head. I’ve always been a political animal and very socially aware of the world around me. Being a creature of New York City, I’ve met many souls from various pockets around the planet, lived with more subsections of race, creed and faith than most meet in a lifetime and have heard their tales – when you hear a taxi driver spill out more truth about the universe than a dozen philosophy professors, it changes you. I’ve always felt that if you could just step into another persons shoes, see what they see – it would do a great deal to improve how our species relates to others of its kind. If a person could see what its like to be told you have cancer, or you just sold a patent worth millions, or have to make a choice that benefits your family but wrecks someone else’s family; to see an opposite political point of view, a different faith, how the world looks if you have autism. I want people to see other points of view, or use this tool to help those with a phobia get over their fears or help people to revisit places lost (can you imagine the joy of being able to walk around your childhood home that had burnt down to the ground?)
This new medium might just be the culmination of a lifetime of my interests; art and computers – 3D animation and Photoshop – video games and experiencing adventures previously unimagined. And I’m so thrilled its happening now and I can be a part of it.
Some days, I’m fortunate to have a babysitter come and watch the kids for a couple of hours, keep them distracted, amused and on nice days, out of the house while I get a little dev work done. I plan out my agenda, get my assets in order so I can be efficient and get as much work done in my little allotted time.
This was not one of those days.
I woke up shortly after 5 AM, sounds of my youngest fussing and my wife tending to him. We have a deal, she gets the kids in the middle of the night, and I let her sleep in on the weekends. After 15 minutes or so of non-stop meltdown, I got up to see if I could help. The missus informs me that our lil’ guy stripped off his diaper, peed all over his crib, blankets and stuffed animals – and is now throwing a fit because he can’t go back to sleep with his soiled bedstuff.
Since she leaves early for work, I had to take junior and calm him and coax him back to sleep, which sorta worked, until his big brother decides to wake up at 6:30. Oh well – at least the babysitter will be here today.
Except, when she gets here, I can’t focus. I’m dragging. Creative impulses are as useless as logical focused thinking when I’m tired. The best I can do is play around with a bunch of mocap files I got off the asset store, to see if I can incorporate them onto MakeHuman .fbx files and if they’ll play nicely with Mechanim & PlayMaker. Sadly, I just can’t get it in gear and instead fiddle around in Photoshop, putting together a graphic design idea I’ve had kicking around in my head for a while:
