Shacktopus Architecture

Real physical tasks are queuing up, but it's a rainy holiday weekend and I find myself more engaged in gonzo engineering than rust patrol and plumbing. It began while sketching a simple controller, and suddenly the whole Shacktopus network design became illuminated in my brain... an area that had grown murky, overwhelmed by too many choices over lo, these many years.

But first, some context from one of the nodes...

Brighteye

I have started working on the assembly that bolts onto the very nose of the bow pulpit, adding 6 inches to the overall length of the boat. This module includes a remote-control 3200-lumen HID spotlight, waterproof color camera, IR emitter, a pair of LED navigation lights, a small microcontroller, 3-axis accelerometer, and a sometimes-relevant PIR motion sensor.

Seasoned readers of my perennial technomadic maunderings may recall the video turret, a project we built at UCSD 14 years ago (!) to allow a single camcorder-grade camera to be aimed in any direction with remote zoom and other controls. It turned into quite an ambitious robotic project, as you can see from the photo below: an embedded FORTH board on the Microship multidrop network took care of motion control, complete with a pen-based graphic user interface on a wireless ruggedized Newton that routed commands through the hub. Ahhh, fond memories.



The irony of that machine was that by the time Wordplay was complete enough to consider actually bolting on the turret, cameras had gotten so cheap that it would have been more sensible to scatter a bunch of them around the boat and use the video crossbar to select the source. That's not quite true, since the steerable camera was higher quality than the cheap bullet-cams of the day, but still...

Well, now I have a big ship and the same twisted geek fantasies, so I've been staring again at the turret. I'd really like to use it, but it just doesn't fit the application (although I am tempted to repurpose it by ripping out the video stuff and installing a directional WiFi antenna, using heading from the rate gyro and signal strength data from the router to keep a distant hotspot centered between half-power points).

But I still need cameras around the boat for video production, live webcam, night-watch, and security. At the moment, with Nomadness parked far away in a marina with significant western exposure, I'd really like to be able to check in during windstorms and keep an eye on my docklines (one brand-new Samson Pro-Set 5/8" nylon bow line chafed through one of its 3 strands in the last storm). Besides, there have been recent thefts over there... I have an Axis 210 webcam in the pilothouse, but I want one on the arch staring down at the cockpit as well as a steerable one on the bow that can let me scan the whole area or peer up and down the dock... emailing me if motion-detection software detects change within a defined region while I'm not watching.

Given a steerable sealed camera on the bow, some other capabilities become available... like being able to flip open a laptop and look around from the cozy berth, capture footage of crazy conditions without trying to fiddle with a camcorder while holding on for dear life, and automatically post shots from any angle while traveling. So I am mounting a waterproof Helmet Camera on a high-intensity discharge Golight with wireless remote, then adding an Arduino board linked to the ship's server for position feedback, power control, and sensors.

The video will be piped through an IP camera server, and a little web app will allow control of the light and camera from a browser. If the camera turns out to be sufficiently IR sensitive, I'll add an emitter for night use; there are port bits left over and plenty of power available, so this thing can grow as needed.

This reflects the fundamental paradigm that has driven most of my projects over the years: leveraging existing good engineering by blending slightly-hacked products into something larger. Life is too short for reinventing wheels, and by factoring an idea into components that map onto off-the-shelf solutions, I can minimize the amount of work necessary to break new ground. The two video turrets are a perfect example: the old one was probably close to a man-year of work; this one will probably be done in a couple of weeks of bench time.

Shacktopus and Datawake Evolution

Names are important, for they create mental "handles" for things. It has thus been frustrating for me to have a rather loose definition of Shacktopus, since it began as a sort of communications laptop, paused for 3 years, then evolved into a more generalized toolset for interacting with a complex mishmash of systems without having to remember the details of every one. I've also been bandying about the Datawake concept, originally hatched as the data collection system that leaves a virtual "wake" of information as the ship moves through time and space (it even had a spinoff a few years ago).

Both of those capabilities are essential to the Nomadness system, and, I would argue, are widely useful. Over the past decade, I have come up with variants on this architecture every time I have had a surge of creativity, a glimpse of new technology, or input from someone with good ideas. The resultant was a bit of a jumble, with a half-dozen microcontroller flavors, something that is always on that serves as a hub, and some kind of server that magically provides a web interface while stuffing things into a database and running small apps for security/watch functions.

Well, now that I'm actually building this, it's time to clear up the ambiguities and get clear on specs, components, protocols, nodes, data types, network tools, and all the rest... beginning with the mapping of linguistic handles onto conceptual subsystems so we can talk about it meaningfully:

Shacktopus is a quirky word that I coined in 2005 (rejecting Winnepacko and Geekpack) to reflect the image of an intelligent multi-armed critter controlling a ham shack... initially conceived as a backpack-scaled QRP station with network access and lots of other tools. There is of course a bit of the maritime imagery here as well, and I am now seeing this as the combination of systems that allows both local and remote interaction with communications, sensing, security, power control, A/V, and other gear. Basically, it is a cushioning layer between the user and a hugely complex mess of stuff. It should never be necessary to drag out manuals in order to operate one's own machines, there's no excuse for a tangle of cables, and a large suite of tools shouldn't come to define one's living space. Think Star Trek (in the later years): the ship has sensors everywhere, and its own distributed intelligence. User interface is smooth and consistent, and unless you are named "Data," you interact with the ship mostly through conversation and occasional finger-tapping on a clean control panel.

Datawake lives in the same overall system environment, but has relatively little to do with humans in its day-to-day operations. It's role is to lay down a detailed history, collecting telemetry from all subsystems, tagging it with time and location, and stuffing it into a database. This becomes useful and interesting in a variety of situations, including environmental observations, failure analysis, security, and power management. Still, it's rather passive compared to Shacktopus — it is the archivist.

Now. What does the hardware look like?

I should preface this with a quick comment on granularity. Occasionally over the years I have built sprawling complex machines, the hardware equivalent of spaghetti code. They were impossible to document, hard to debug, and unless used consistently tended to become confusing over time... in some cases, with front-panel reassignments and extensive back-door kluges. But that was the way things were done back in the day; power supplies and packaging were expensive, so you just crammed everything into one box with a lot of front-panel controls to sort it all out:

Winnebiko II console, circa 1988

Then, in the BEHEMOTH bicycle era, I started using FORTH boards from New Micros. My hardware was still complicated, but I was learning to think differently about code: clear little modules called words, added to the existing ones in a dictionary, with higher-level functions being expressed in terms of lower-level ones in this "extensible" language... usually only a few lines of code each. Over time, I started looking at hardware in the same light, appreciating simple gadgets that do one thing perfectly, with well-defined inputs and outputs and no back doors, side-effects, or shared functions.

Of course, we now have object-oriented programming, which carries this much further (though with more of a "big system" flavor). It's part of our culture, and rightly so. What is less known is that a similar phenomenon has been growing in the world of chips; you can still load up on surface-mount devices and make custom boards, of course, but a number of vendors now offer a very wide range of little gizmos that make complex devices easy to use and hide many of the details. These range from prototype-friendly breakout boards for single chips to low-cost subsystems like USB data-collection modules that would take months to develop.

The difference may be moot at the big-picture level, but what this means is that one can assemble just about anything these days without having to build custom printed circuit boards (unless, of course, volume production requires doing so for cost-minimization).

Aboard Nomadness, this is a lifesaver, and means there should be a fairly short path to initial results.

System Architecture in a Nutshell

So, with all that preface behind us, how does this work? Easy...

One machine will be on all the time, most likely a small Linux box. I'm still considering the Mini-ITX and its variants, but it may be that starting with a finished product made for battery operation is the shortest path to a power-efficient system that works out-of-the-box. I'm looking at the class of subnotebooks called netbooks, many of which ship with Linux and run on the low-power Atom processor from Intel with SSD file storage. Perfect.

Broken in two with the LCD panel-mounted, this yields a net-savvy server with robust tools. A power drain of 2-3 watts I can live with; although I'd love to use a Mac Mini in this role, it's much more power hungry at 20-30 watts (still light compared to desktops, of course, but see the excellent research conducted by AMUG on the subject).

The real lifesaver here that should make development easy is cheap USB stuff. All the interfaces I need (except for an IP video server directly on ethernet) exist in USB flavors, and one big pile of hubs will provide all the connectivity I need for the I/O hardware:

1. Latching relays for configuration management, audio routing as needed for comms (in addition to the console mixer), video routing (about a dozen sources and 6 sinks), and some local power control. (Relays don't switch in sync with the video retrace interval like my old Mitel 88V32-based Vixbar, of course, but for this application, I'll live with the glitch.) My earlier crossbars allowed complete generality; this approach is sparse-matrix but the straight-through connection is signal-agnostic and reconfigurable.
   
2. Digital output bits connected to solid-state relays for power control... standard industrial DIN-rail packaging from Phoenix Contact.
3. Digital input bits (local to the console) for power distribution mapping and other close-in status detection.
4. A bank of retro serial interfaces for RS-232 gadgets like a text-to-speech board, ham radio control ports, a TNC for datacomm, and a bunch of Arduino microcontrollers scattered around the boat. These include a DTMF decoder for remote control via UHF handheld, the light/camera/sensor module on the bow, hatch security sensors, water and air system monitors, bilge water and pump-cycle monitor, cockpit security, remote front-panel interfaces, and so on... possibly even one of the old Auxbar units.
   
5. An XBee radio for the two Arduinos connected by wireless links: dinghy security and an OLED-touchscreen handheld unit (though the latter might be more easily accomplished with an off-the-shelf PDA).
   
6. National Instrument modules for analog/digital/pulse data collection.
7. Maretron USB100 gateway, slurping in all the PGNs in the ship's NMEA 2000 network (rate gyro compass, GPS, derived navigation data, rudder angle, fuel level sensors, engine and fuel flow data, wind speed and direction, depth, outside temperature, power system data).

With one fell swoop and no extra computers, we've just created an environment that accommodates every data type on the boat. Video sources can be switched to a server and viewed in a browser, radios can be reconfigured complete with audio and PTT steering, all data can be collected and stashed in a database-backed website, remote microcontrollers can be queried or mirrored to a terminal window, and (assuming I can figure out the undocumented N2K stream that currently only ships with a Windows client) we can create a multifunction display for the ship nav systems and engine.

And, my favorite part: it's all off-the-shelf parts, most of which are cheap. That translates into incremental development that frequently injects positive feedback to restart the attention-span timer... while still moving me in the same direction for a long time (the key to getting Big Projects done).

While writing this, it occurred to me that I should start posting detailed tech info to the Joomla-based Nomadness site. The split between this blog and that yet-sparce pile of content is along the vague boundary of narrative/noodling and how-to documentation. This isn't the latter quite yet, but pieces are now on order and we should see blinkies very soon.

Cheers...
Steve

Consoling Thoughts

The modular approach to ship system fabrication is starting to manifest itself as little pools of parts in the lab... good thing we resurfaced the crusty old particle-board workbenches a few months ago! The current lab-based projects are:

1. Console (communications gear, audio, geekstuff)
2. Waterworks (fresh-water processing)
3. Bow Module (steerable spotlight/video/sensors & LED navlights)
4. Field Radio System (QRP, PSK31, antenna, solar panel, batteries)

This makes sense from a development standpoint, given that the boat is an hour and a half away. Weeks slip by as I shuttle back and forth, piddling at measurements and brainstorming, fixing things, and getting immersed in whatever I didn't go there to work on. Like, how did I end up in this huge water project? Oh yeah, I remember... I opened the space above the starboard fuel tank to install the Wema level sensor, thought I'd check to see if the watermaker would fit there, didn't like it, considered alternatives, and ended up thinking it would be great on the aft head wall along with a filtration system. So now I'm deep into fittings, rotameters, pressure regulation, TDS monitoring, and a coherent user interface involving over a dozen valves.

And no, I haven't installed the Wema sensor in the starboard tank yet.

This business of being finite is no less annoying than it was during my more youthful technomadic projects, but of course it is harder now that the toys are growing bigger as the protoplasm grows creakier. I'm not looking forward to some of the projects that are more essential and less alluring... like dangling over the stern with an angle grinder to take dinghy-wounds down to bare metal before infinitely patient rust succeeds to the point where I have to throw another log on to compensate for additional breezes through the cabin.

Much easier to research gadgets online, refine system drawings, and spin 'round in my chair now and then to play Worrisome Blues on the piano...

<ahem> Actually, that is not so far out of context as it may seem... there's real pleasure in this new musical pursuit, now 2 years along. I've played flute all my life, but never with any formality (and certainly never with sheet music, except as an abstract graphic crutch for something I could already play by ear... "oh, it goes up here"). I think one of the pleasures in this nascent ivory tickling is the engagement of wetware bits that are not otherwise being used much; it's a real stretch for these slothful old neurons to get my fingers moving in synch with the chicken-scratchings on the page, but the result is startlingly visceral and I refine it obsessively.

Kind of like sailing. Music is non-intellectual yet immediate, and it wraps around such an elegant conceptual infrastructure that every new discovery is an "aha" moment that reveals the next puzzle in the midst of the rush from clearing up the previous one.

Mathematics is music for the mind. Music is mathematics for the soul.
- Anonymous

Since my "life's work" seems to be characterized by blending all my passions into some kind of nomadic quest, it is only natural that this is going to have to be shoehorned in. The communications console is the logical place, as it already incorporates both computer-controlled crossbar switching (like my old Auxbar design) and a sexy audio mixer. I don't really know where the piano will go; I'll probably end up buying a cheapie and hacking it to fold in half for stowage. That's very much back-burner at the moment, but inevitable.

Audio, Video, Power, Data, Sensors...

The photo below is Grand Central Station from the Microship project, consisting of audio, video, and serial crossbar networks running in networked FORTH 68HC11 boards from New Micros:


I've been tempted to re-use this (among other things, it works, which is a huge plus), but in the interest of scalability I'm using USB-controlled latching relay boards for much of the signal routing. They have near-zero insertion loss, and can be controlled by a small always-on micro, with the boundaries separating audio, video, PTT, and even antique RS-232 endlessly flexible (unlike the strict "data types" imposed by the crossbar design above). It's interesting, though, to see that this problem remains central even 15 years later; part of the whole Shacktopus vision is a unifying layer that erases boundaries between gadgets, and I have yet to find a more effective approach than simply folding everything into a consistent switching matrix with a suitable UI on top.

This brings me back to the console, which has been a sort of mental catch-all zone for yet-to-be installed geekery. Now that I'm laying out panels with actual-size radio templates and the like, I'm finding myself doing a use-case analysis to figure out what is really needed (and where). This is the payoff from the shakedown cruise: such noodlings are no longer abstract.

There are really only a few major "clusters" when it comes to interacting with the techie bits...

1. Navigation: Emphasis here is a decent chartplotter, and (if underway instead of voyage planning) close integration with autopilot, manual steering controls, radar, and general instruments... with ready access to power management and marine VHF radio.
2. Playing radio: Focus on communications, whether ham radio, marine SSB nets, email, shortwave listening, or just tinkering.
3. Audio/Video: Less a mode than a diverse toolset, this includes stereo gear, the piano, mixer, camera suite, signal routing hardware to steer sources to local monitors or into the server for net access, recording tools, and so on.
4. Power: Another "infrastructure" item, including AC and DC controls, battery management, solar panels, generator, shore power, and all the little chargers for hand-held gear.
5. Hacking: The lab, with basic instruments, prototyping, debugging tools, and (importantly) reasonably central access to all signals, sensors, and power-control lines on the ship..
   
6. Shacktopus: Finally, the layer that binds all the rest together... the tools that render everything coherent from any local operating position, including (with relaxed expectations and robust authentication) a remote browser anywhere on the Internet and voice I/O via radio. This includes both "big iron" that is browser-capable, and lightweight always-on micros that maintain a consistent presence at both the back end (data collection) and front end (user interface).
   

The fun puzzle is mapping these very overlapping and intertwined conceptual spaces into actual hardware involving scarce console real estate and bundles of cable. Until recently, I have been assuming that the existing "inside helm" would remain more or less inviolate territory, and the "geeky bits" would all be compressed into what was originally the nav station. Devices that bridge these spaces, like the main Mac LCD, would be on a swing-arm mount to allow use wherever I happen to be sitting.

I've run this photo before, but it's worth including here for visual context:



(That image is a year out of date, lacking the wood stove and the new instrument cluster at the helm... but it is the only wide-angle shot of the pilothouse I have. It was taken by the broker, Cindy Mettler, whose job it was to make the boat look spacious for the Yachtworld listing.)

Now that I've spent 2.5 months aboard, I have a more practical view of how this needs to work. The comm console (past the galley on the left) should incorporate radios, audio stuff, and tinkering tools; that's the most open work desk, and is the obvious spot for front-panel-intensive equipment that is not directly involved in boat operations.

The inside helm, at the right surrounding the big chair, actually has a lot of open space. Everything below the DC breaker panel is obsolete and going away, and the upper right quadrant of the helm itself (above the Yanmar diesel control panel that is peeking around the chair) is a Robertson autopilot that has already been replaced. Looks like the perfect spot for a high-brightness LCD owned by a Mac Mini, running MacENC as well as a browser for access to the ship server. The outside helm, where I actually spend most of my time while piloting, will have an appliance chartplotter for maximum turnkey reliability, but the Mac solution is a much more flexible use of console real-estate below.

If I need system goodies anywhere else, then I can use a laptop. There's a hotspot in the boat anyway, so it doesn't really matter where I'm sitting.

This approach is very liberating, and is a clean break between ship fundamentals and the overlay of gizmology. It also allows better spillover into underutilized console space above the table to starboard, and cabling over there is much easier (especially where power is involved). This is going to be fun.

The Kicker

One quick bit of news before I wrap this up... I finally broke down and converted a month's worth of eBayage into auxiliary propulsion for Nomadling (the dinghy), in the form of a Yamaha 2.5 horsepower 4-cycle outboard. I know. It's hard to be an electric-vehicle purist when I already drive an 18-ton sailboat with two big diesels on board, but I need to reliably make a 10-mile commute each way between lab and moorage to avoid the 54-mile alternative via truck. A quart of gas should do it while being more fun in the process (assuming I'm not trying to haul the table saw), and it's really a sleek little contraption... but I'll reserve judgement until I actually try it in open water sometime in the next few days.

The smaller the boat, the bigger the adventure!

Yarrh,
Steve

Waterworks, Shacktopus, and Simplicity

Work trip #3 was a short one, aborted by bad weather and flagging motivation. I showed up after dark with big companionway-hacking plans for the next day, hauled my table saw to the dock and covered it with plastic to keep condensation at bay, stayed up late faffing around online, then woke pre-dawn to a full gale with the Bosch threatening to take a flying leap into the drink. I lashed 'er down and tried to get back to sleep, but no luck. It rained all day, and the enormity of the whole project weighed on me until I shrugged and made the hour and half commute back home.

Sometimes that's just the way it goes.


Indeed, the hardest part of this is keeping a vision of the recent teaser voyage in the foreground of my consciousness whilst grappling with a lab full of poignant dusty reminders of projects unfinished, short cold days of a northwest winter, and winter moorage so far away that going to the boat is an event.

What this means in practice is that I have to take a modular approach, working on subsystems in the lab and then hauling them aboard to deal with integration. That's necessary anyway, given the minimal shop facilities on the boat, but it adds another layer of synchronization if I am to avoid surprises. There are three major modules in progress: communication console, outside helm nav station, and the waterworks.

Feeding the Tanks

As usual, it started simply. Last week, I took the Katadyn 40E watermaker to the boat in order to confirm the mounting location... assuming that it would just go in the big hole where the old one came out. It did fit easily, but I wasn't happy about the thought of servicing salt-water prefilters in the same equipment bay that includes the 115-volt AC breaker panel, so looked instead at the space under the galley sink. That could work, though it would be an awkward three-dimensional puzzle to install and will be very uncomfortable if I ever need to remove the electric pump and operate it manually due to a failed power system (one of my main reasons for choosing that model).

Right on the other side of the wall, however, is the aft head compartment, where the decommissioned Bosch propane-fired demand water heater is still hanging (it is being replaced by an Isotemp mounted behind the shower compartment). The available space is on the order of 24 by 34 inches, with a good 3-4 inches of depth before it would start to become annoying while interacting with the Lavac. Hmmm.

Suddenly the water-processing system snapped into focus. The reverse-osmosis watermaker and its prefilter will go there... along with a Water Fixer ultraviolet purification system. The old Bosch exhaust opening will carry a dock-water fitting and entry point for jugs or rainwater, and a system of valves will allow cleaning the water enroute to the tanks, or from the tanks on the way to the spigots around the boat. It will even support "water polishing" as I do with the diesel fuel (pumping from one tank to another through a Racor filter), and four channels of total dissolved solids monitoring will give me a quick look at watermaker output, ship pressure water, incoming dock water, and the super-clean stuff after the point-of-use filter at the drinking tap.

In the time-honored tradition of creeping featuritis, this continued to grow until it reached absurdity, then was pruned by the usual techniques of factoring, sanity-checking, and tossing out superfluities. It now includes a (locked) valve to permit slurping in raw water when I'm in river systems, a high-silt booster pump and prefilter for dirty brine, a way to route filtered dock pressure water directly to taps instead of into the tanks, and various other features. Naturally, the biggest design challenge is making sure that the user interface (14 valves) is clear enough to make sense some spring day five years from now when something is amiss and I have not just spent hours staring at plumbing diagrams.

This has to be self-documenting, so a hinged cover panel will carry a big clear graphic that makes it obvious what's going on. More webness to come on this topic as it develops.



Shacktopus

I've dusted off the old equipment box from Bubba-the-kayak, and am installing the Yaesu FT-817ND ham rig and the NUE-PSK modem. This already has 24 amp-hours of battery, sealed connectors, a dedicated 30-watt solar panel with charger, and related goodies... so in the salty Nomadness context this will likely become part of the new Shacktopus system (or at least a dedicated field radio station independent of the rigs built into the ship).

Shacktopus began as a sort of "communications laptop" that was under intensive development here in 2005 until my dad passed away and necessitated a 6-month expedition to shut down the old homestead in Kentucky. A small Linux board and a lightweight micro presented a webbish interface to a rather large suite of tools, including radio front end and a variety of sensors... and it could be accessed via a handheld radio using DTMF commands and synthesized voice response as well as a PDA or laptop. Reminiscent of the BEHEMOTH and Microship projects, audio and serial routing allowed all devices to appear as a single, simple user interface (here's the drawing, although the system was never completed).

I no longer need a backpack rig like that, but the design problem on Nomadness is very similar: lots of diverse capability that has to be accessible from a variety of places with a minimum of hardware. Since a lot of engineering went in to Shacktopus, I've been thinking quite a bit lately about how to dovetail it into the new ship while keeping the boundaries clear enough to allow product spin-off down the road. The key, as usual, is modular design.


The basic problem is familiar to almost any radio geek: lots of rigs and gadgets, all with their own ideas about power, speakers, microphones, operating procedures, and front panels. The result is usually a sprawling "ham shack" with dangling mikes, a snarl of cables, custom switch boxes and patch panels to multiplex scarce resources, an overall feeling of clutter, and a high likelihood of pilot error.

In a mobile environment, this is simply unacceptable... yet even here in 2008 there is no standard "bus" along the lines of NMEA 2000 that can tie radio modules into a single integrated environment. Most modern rigs do have computer-control capability... and there is even a comprehensive set of API tools called hamlib that creates a layer of abstraction between vendor-specific stuff and the broad concepts of radio operation. In Shacktopus, this is carried much further, incorporating Wi-Fi, local environmental and security sensors, mode-specific tools like PSK-31, location-aware applications, graceful adaptation to available network resources, power management, and so on... exactly what I want on the boat.

So the upshot of all this is that it has finally become clear how the Shacktopus concept maps onto the Nomadness project. Everything on the ship needs to be accessible at multiple levels: local dead-simple interface that works when the geeky stuff is broken or unfinished, web-accessible toolset that can be reached within the LAN or (more slowly) from afar, voice interface that can allow at least some functionality from any hand-held radio, and a clean local operating console that hides as much complexity as possible while encouraging application-specific activity. The waterworks mentioned above is a good example of the latter: associated with my drawing of the system is a chart that relates each operating mode to a set of valve positions (open, closed, or don't-care). I don't want to have to figure that out every time, for therein lies madness; the user interface needs to be modal and remotable, yet open enough to allow hacks and work-arounds when something goes awry.

Simplicity

Somewhere under all this is a sailboat, of course, and I haven't forgotten that. Old salts roll their eyes at this apparently gratuitous geekery and remind me that simple is always better on a boat.

Believe it or not, I agree completely.

The basic premise of this whole project is not the addition of complexity... that would be a failure. Instead, it is a layering of tools to maintain the simplicity of a sailing vessel while adding a range of useful capabilities that would normally be considered impossible in such a context. If that can be done without cluttering the experience with constant tinkering, then we will have succeeded.