Shacktopus Architecture

Real physical tasks are queuing up, but it’s a rainy holiday weekend and I find myself more engaged in gonzo engineering than rust patrol and plumbing. It began while sketching a simple controller, and suddenly the whole Shacktopus network design became illuminated in my brain… an area that had grown murky, overwhelmed by too many choices over lo, these many years.

But first, some context from one of the nodes…

Brighteye

I have started working on the assembly that bolts onto the very nose of the bow pulpit, adding 6 inches to the overall length of the boat. This module includes a remote-control 3200-lumen HID spotlight, waterproof color camera, IR emitter, a pair of LED navigation lights, a small microcontroller, 3-axis accelerometer, and a sometimes-relevant PIR motion sensor.

Seasoned readers of my perennial technomadic maunderings may recall the video turret, a project we built at UCSD 14 years ago (!) to allow a single camcorder-grade camera to be aimed in any direction with remote zoom and other controls. It turned into quite an ambitious robotic project, as you can see from the photo below: an embedded FORTH board on the Microship multidrop network took care of motion control, complete with a pen-based graphic user interface on a wireless ruggedized Newton that routed commands through the hub. Ahhh, fond memories.


The irony of that machine was that by the time Wordplay was complete enough to consider actually bolting on the turret, cameras had gotten so cheap that it would have been more sensible to scatter a bunch of them around the boat and use the video crossbar to select the source. That’s not quite true, since the steerable camera was higher quality than the cheap bullet-cams of the day, but still…

Well, now I have a big ship and the same twisted geek fantasies, so I’ve been staring again at the turret. I’d really like to use it, but it just doesn’t fit the application (although I am tempted to repurpose it by ripping out the video stuff and installing a directional WiFi antenna, using heading from the rate gyro and signal strength data from the router to keep a distant hotspot centered between half-power points).

But I still need cameras around the boat for video production, live webcam, night-watch, and security. At the moment, with Nomadness parked far away in a marina with significant western exposure, I’d really like to be able to check in during windstorms and keep an eye on my docklines (one brand-new Samson Pro-Set 5/8″ nylon bow line chafed through one of its 3 strands in the last storm). Besides, there have been recent thefts over there… I have an Axis 210 webcam in the pilothouse, but I want one on the arch staring down at the cockpit as well as a steerable one on the bow that can let me scan the whole area or peer up and down the dock… emailing me if motion-detection software detects change within a defined region while I’m not watching.

Given a steerable sealed camera on the bow, some other capabilities become available… like being able to flip open a laptop and look around from the cozy berth, capture footage of crazy conditions without trying to fiddle with a camcorder while holding on for dear life, and automatically post shots from any angle while traveling. So I am mounting a waterproof Helmet Camera on a high-intensity discharge Golight with wireless remote, then adding an Arduino board linked to the ship’s server for position feedback, power control, and sensors.

The video will be piped through an IP camera server, and a little web app will allow control of the light and camera from a browser. If the camera turns out to be sufficiently IR sensitive, I’ll add an emitter for night use; there are port bits left over and plenty of power available, so this thing can grow as needed.

This reflects the fundamental paradigm that has driven most of my projects over the years: leveraging existing good engineering by blending slightly-hacked products into something larger. Life is too short for reinventing wheels, and by factoring an idea into components that map onto off-the-shelf solutions, I can minimize the amount of work necessary to break new ground. The two video turrets are a perfect example: the old one was probably close to a man-year of work; this one will probably be done in a couple of weeks of bench time.

Shacktopus and Datawake Evolution

Names are important, for they create mental “handles” for things. It has thus been frustrating for me to have a rather loose definition of Shacktopus, since it began as a sort of communications laptop, paused for 3 years, then evolved into a more generalized toolset for interacting with a complex mishmash of systems without having to remember the details of every one. I’ve also been bandying about the Datawake concept, originally hatched as the data collection system that leaves a virtual “wake” of information as the ship moves through time and space (it even had a spinoff a few years ago).

Both of those capabilities are essential to the Nomadness system, and, I would argue, are widely useful. Over the past decade, I have come up with variants on this architecture every time I have had a surge of creativity, a glimpse of new technology, or input from someone with good ideas. The resultant was a bit of a jumble, with a half-dozen microcontroller flavors, something that is always on that serves as a hub, and some kind of server that magically provides a web interface while stuffing things into a database and running small apps for security/watch functions.

Well, now that I’m actually building this, it’s time to clear up the ambiguities and get clear on specs, components, protocols, nodes, data types, network tools, and all the rest… beginning with the mapping of linguistic handles onto conceptual subsystems so we can talk about it meaningfully:

Shacktopus is a quirky word that I coined in 2005 (rejecting Winnepacko and Geekpack) to reflect the image of an intelligent multi-armed critter controlling a ham shack… initially conceived as a backpack-scaled QRP station with network access and lots of other tools. There is of course a bit of the maritime imagery here as well, and I am now seeing this as the combination of systems that allows both local and remote interaction with communications, sensing, security, power control, A/V, and other gear. Basically, it is a cushioning layer between the user and a hugely complex mess of stuff. It should never be necessary to drag out manuals in order to operate one’s own machines, there’s no excuse for a tangle of cables, and a large suite of tools shouldn’t come to define one’s living space. Think Star Trek (in the later years): the ship has sensors everywhere, and its own distributed intelligence. User interface is smooth and consistent, and unless you are named “Data,” you interact with the ship mostly through conversation and occasional finger-tapping on a clean control panel.

Datawake lives in the same overall system environment, but has relatively little to do with humans in its day-to-day operations. It’s role is to lay down a detailed history, collecting telemetry from all subsystems, tagging it with time and location, and stuffing it into a database. This becomes useful and interesting in a variety of situations, including environmental observations, failure analysis, security, and power management. Still, it’s rather passive compared to Shacktopus — it is the archivist.

Now. What does the hardware look like?

I should preface this with a quick comment on granularity. Occasionally over the years I have built sprawling complex machines, the hardware equivalent of spaghetti code. They were impossible to document, hard to debug, and unless used consistently tended to become confusing over time… in some cases, with front-panel reassignments and extensive back-door kluges. But that was the way things were done back in the day; power supplies and packaging were expensive, so you just crammed everything into one box with a lot of front-panel controls to sort it all out:

Winnebiko II console, circa 1988

Then, in the BEHEMOTH bicycle era, I started using FORTH boards from New Micros. My hardware was still complicated, but I was learning to think differently about code: clear little modules called words, added to the existing ones in a dictionary, with higher-level functions being expressed in terms of lower-level ones in this “extensible” language… usually only a few lines of code each. Over time, I started looking at hardware in the same light, appreciating simple gadgets that do one thing perfectly, with well-defined inputs and outputs and no back doors, side-effects, or shared functions.

Of course, we now have object-oriented programming, which carries this much further (though with more of a “big system” flavor). It’s part of our culture, and rightly so. What is less known is that a similar phenomenon has been growing in the world of chips; you can still load up on surface-mount devices and make custom boards, of course, but a number of vendors now offer a very wide range of little gizmos that make complex devices easy to use and hide many of the details. These range from prototype-friendly breakout boards for single chips to low-cost subsystems like USB data-collection modules that would take months to develop.

The difference may be moot at the big-picture level, but what this means is that one can assemble just about anything these days without having to build custom printed circuit boards (unless, of course, volume production requires doing so for cost-minimization).

Aboard Nomadness, this is a lifesaver, and means there should be a fairly short path to initial results.

System Architecture in a Nutshell

So, with all that preface behind us, how does this work? Easy…

One machine will be on all the time, most likely a small Linux box. I’m still considering the Mini-ITX and its variants, but it may be that starting with a finished product made for battery operation is the shortest path to a power-efficient system that works out-of-the-box. I’m looking at the class of subnotebooks called netbooks, many of which ship with Linux and run on the low-power Atom processor from Intel with SSD file storage. Perfect.

Broken in two with the LCD panel-mounted, this yields a net-savvy server with robust tools. A power drain of 2-3 watts I can live with; although I’d love to use a Mac Mini in this role, it’s much more power hungry at 20-30 watts (still light compared to desktops, of course, but see the excellent research conducted by AMUG on the subject).

The real lifesaver here that should make development easy is cheap USB stuff. All the interfaces I need (except for an IP video server directly on ethernet) exist in USB flavors, and one big pile of hubs will provide all the connectivity I need for the I/O hardware:

  1. Latching relays for configuration management, audio routing as needed for comms (in addition to the console mixer), video routing (about a dozen sources and 6 sinks), and some local power control. (Relays don’t switch in sync with the video retrace interval like my old Mitel 88V32-based Vixbar, of course, but for this application, I’ll live with the glitch.) My earlier crossbars allowed complete generality; this approach is sparse-matrix but the straight-through connection is signal-agnostic and reconfigurable.
  2. Digital output bits connected to solid-state relays for power control… standard industrial DIN-rail packaging from Phoenix Contact.
  3. Digital input bits (local to the console) for power distribution mapping and other close-in status detection.
  4. A bank of retro serial interfaces for RS-232 gadgets like a text-to-speech board, ham radio control ports, a TNC for datacomm, and a bunch of Arduino microcontrollers scattered around the boat. These include a DTMF decoder for remote control via UHF handheld, the light/camera/sensor module on the bow, hatch security sensors, water and air system monitors, bilge water and pump-cycle monitor, cockpit security, remote front-panel interfaces, and so on… possibly even one of the old Auxbar units.
  5. An XBee radio for the two Arduinos connected by wireless links: dinghy security and an OLED-touchscreen handheld unit (though the latter might be more easily accomplished with an off-the-shelf PDA).
  6. National Instrument modules for analog/digital/pulse data collection.
  7. Maretron USB100 gateway, slurping in all the PGNs in the ship’s NMEA 2000 network (rate gyro compass, GPS, derived navigation data, rudder angle, fuel level sensors, engine and fuel flow data, wind speed and direction, depth, outside temperature, power system data).

With one fell swoop and no extra computers, we’ve just created an environment that accommodates every data type on the boat. Video sources can be switched to a server and viewed in a browser, radios can be reconfigured complete with audio and PTT steering, all data can be collected and stashed in a database-backed website, remote microcontrollers can be queried or mirrored to a terminal window, and (assuming I can figure out the undocumented N2K stream that currently only ships with a Windows client) we can create a multifunction display for the ship nav systems and engine.

And, my favorite part: it’s all off-the-shelf parts, most of which are cheap. That translates into incremental development that frequently injects positive feedback to restart the atte
ntion-span timer… while still moving me in the same direction for a long time (the key to getting Big Projects done).

While writing this, it occurred to me that I should start posting detailed tech info to the Joomla-based Nomadness site. The split between this blog and that yet-sparce pile of content is along the vague boundary of narrative/noodling and how-to documentation. This isn’t the latter quite yet, but pieces are now on order and we should see blinkies very soon.

Cheers…
Steve

This entry was posted in Uncategorized and tagged , , , . Bookmark the permalink.

4 Responses to Shacktopus Architecture

  1. Anonymous says:

    Metaphor objection: “Still, it’s rather passive compared to Shacktopus — it is the librarian.”

    Datawake is the catalog librarian. Shacktopus is the reference librarian/information scientist.

    Mary Anne in Kentucky (former librarian)

  2. Ahhh, that makes sense. I'll fix it! Thanks…

    -Steve

  3. Anonymous says:

    Steve,

    Would you speak a bit about how much power do you currently use … and have?

    I think it would be interesting to see how much is dedicated to general housekeeping issues (water maker, refrigeration, hot water, etc), what is considered to be general boat-related items (SSB, HAM, electric windlass) and tech items (cameras, EVDO, etc).

    Also, I love to know how you are getting the power (solar, wind, generator, methanol fuel cells)

    Be well,

    Maren

  4. Thank you for the request, Maren. My friend Tim Nolan is arriving in January to help build the new power system (based on Outback equipment), and at that time I’ll do some detailed pieces on the subject. I’ll keep your specific questions in mind!

    Cheers and best wishes,
    Steve

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>