Servo Endurance and Failure

The main steering servo that has been used in Voyager 2 has been the Hitec HS-322HD.

Several of these servos have been used in the life of the Voyager 2.

The original steering servo used in lake conditions, from 2016 was being retired in late 2019, prior to commencing ocean voyages, and stored away for possible use in less critical projects.

Hitec HS-322HD

A few years later in January 2023 this servo tested for its endurance by exercising without load until it failed.

It completed around 4 million movements before failure.

The failure occurred in the brushes of the DC motor .

The image below shows two motors from HS-322HD servos where the brushes had failed.

In both cases the filed brushes lead to a failure of FET drivers in the H-Bridge.

The brush housing on the left is from the motor tested to failure without load, to 4 million operations.

The one on the right failed at sea, leading to the stranding of the Voyager near Cape Schank in April 2024.

Failed DC Motor Brushes

Servo Driver board showing burnt-out FET drivers.

The steering servo on board Voyager for the 6th voyage in Bass Strait, failed in April 2024.

It failed after an estimated 280,000 operations. This is well short of the anticipated 4 million operations from the previous no load endurance testing.

The plan moving forward is to investigate the availability and suitability of servos that utilize brushless motors.

The Hitec HS-322HD has a torque rating of 3kg at 1cm.

The smallest brushless servo seems be rated at 10kg at 1cm.

100 days on a Beach - 6th Bass Strait Voyage

100 days on a Beach and a broken leg - 6th Bass Strait Voyage 

The 6th voyage on Bass strait for Voyager 2.7, commenced from Torquay Fishermen's Beach at around 4:45pm April 17th  2024.

Conditions were good.

Pre-launch at Torquay Fishermen's Beach 

But, the next day after around 28 hours sailing, Voyager appeared to suddenly deviate from course and drift to coast under the influence of the South-Westerly  wind, and eventually washed up in an isolated cove near Cape Schank, Victoria.

Track reported by the Satellite tracking showing the apparent time of failure.

The coast near Cape Schank is difficult to access so recovery will be difficult.

The coast line is mostly cliffs around 30metres high. It is mostly private rural residential land or farm land.

I attempted to gain access by following a gully down to the coast, but during the hike, I fell and broke my leg. This put me out of action for around 10 weeks.

Coast near Cape Schanck where Voyage come ashore.

The Satellite locator beacon continued to function for several weeks until the batteries finally ran down. the last signal was received at 9pm 22/5/2024, after 35 days. 

While recovering from my broken leg, I formed plans to recover Voyager by water, using my kayak.

The nearest practical location to commence the recovery trip was from Flinders Ocean Beach. This beach had easy access by car, and is protected by offshore reefs.

This would result in a journey 4 miles WSW toward Cape Schank, to the location of  Voyager.

Planned recovery journey in a kayak from Flinders Ocean Beach.

Then on 6th July 2024, after 80 days, a friend flew his light aircraft over the location of Voyager, took photographs, that showed the boat was still in the cove.

It is visible in the image below as the white dot on the beach above the high water mark.

View of Voyager from the air.

I planned my recovery trip using the kayak. I practiced paddling an equivalent distance in the Yarra River near to home to ensure that I could complete the journey in Bass Strait.

I needed calm conditions in Bass Strait, with out too much wind or swell, and not too much rain.

I made two trips to launch location, 1.5 hours drive away, but did not proceed because I was not happy about the conditions. 

The third time the conditions were good to proceed, and on 30th July 2024 at around 10am I started paddling.

The trip in the kayak was around 2 hours, and timed for low tide at the destination.

Voyager was found high up on the pebble beach as expected.

The beach consisted of wave washed rocks around 6 inches across. it was difficult to walk on.

Voyager was found in good condition, considering it had been on the rocky beach for over 100 days by now.

The electronic housing was intact, and the electronics were in good condition.

 

She had been on the beach for 100 days, or 3 1/2 months.

Voyager 2.7 as found after 100 days

Voyager was strapped on to the kayak, and I headed home as quickly as possible, retracing the 4 miles back to Flinders Ocean Beach, another 2 hours.

After recovery, heading home.

What caused the failure ?

At first I thought the failure was due to a mechanical problem with the Wing Sail. 

The Tail section was completely missing, including the stainless steel screws, leaving clean 2mm threaded holes in the carbon fibre tail booms.

The electronics appeared to be in good. 

I powered up the electronics while preparing to rebuild and relaunch Voyager.

But I was surprised to find the main steering servo was not operating, despite appearing to be in good condition.

The servo was disassembled and revealed an obviously burnt-out FET driver transistor.

This is one of the four FET driver transistors that form the H-bridge DC motor driver.

Steering Servo showing the burnt FET Motor Driver.

Further disassembly revealed that the brushes on the DC servo motor had failed.

The brushes each consist of around 4 fingers of brass or bronze.

The image below shows one of the fingers has been displaced and was on the wrong side of the commutator.

This would have shorted-out the motor causing the failure of the FET driver transistor.

Servo Motor with failed brush

I had previously studied the log files on the SD Card, but had not looked carefully at the current consumption.

After reviewing the current consumption, the time of failure of the servo become clear, and is shown in the image below.

the servo failed at around 80,000 seconds into the voyage, or around 22 hours.

Plot of Current versus time showing failure near 80,000seconds.

The failure time was reviewed against the track taken by Voyager, and is shown in the image below.

More detailed review of the logs around this time showed, the vessel lost steerage and its heading appeared uncontrolled from the time onwards. 
It was quite a coincidence that the wind was running almost directly down the desired course.

This continued for around 4 hours, until the wind changed direction and blew the vessel toward land.

Actual steering servo failure time 

The servo had failed prematurely. 

It was a Hitec HS-322HD.

Previous servo endurance testing had yielded around 4 million operations before failure, due to the failure of  the motor brushes.

This servo failed at around 280,000 operations. well below expectations.

Typical servo usage seems to be around 50,000 movements per day.

Servos will be covered in more detail in a future article.

The plan is to investigate brushless servos that should have a much greater lifespan.

 Salt Water and Electronics - 5th Bass Strait Voyage

After quickly recovering from the prior launch and recovery on Friday 22/3/2024, the vessel was ready for another launch 11 days later on Thursday 2/4/2024, the 5th voyage on Bass Strait.

Following the prior launch and recovery on the Friday, the compass had been replaced and recalibrated yielding much better accuracy. The aim was specifically to improve the determination of the True wind Direction, which contributed to the failure as discussed in the last post.

Om that Friday night, the vessel had been recovered high and dry on the beach after it had been washed ashore, through the small surf.

There was little damage, but the sail and its electronics would have been submerged in the breaking waves on beach.

It was considered that there were no electronic problems with the Sail, and so the vessel was launched again on Thursday 2/4/2024.

Launch from Torquay 2/4/2024 at 8:15pm, soon after sunset

It sailed well, in the light to moderate conditions for the next 16 hours, and almost 15 miles down the course, at which point it failed to proceed down the course, and to started to drift back toward the launch place.

The conditions at that point were very calm.

The vessel drifted back to the west for 2 days and returned coincidentally almost back to the launch point. 

As the boat approached the shore it was possible to interrogate it using the LORA telemetry, when it was within around 2km of the headland.

It was possible to confirm that most of the components were operational.

However the wing sail had not responded to regular polling, since the time of failure, 2 days earlier.

This implied that the Wing Sail controller had failed.

The vessel was safely brought to shore by swimming out to retrieve it, to avoid it entering the breakers. 

Safely retrieved, and brought to shore before entering the breakers.

It was noted that wing sail servo was not operational, and was in the neutral position.

This suggests that it is likely that the last action performed by the controller was to partially reboot, and center the servo. If it had failed without involving a reboot, the servo would most likely have been set to one side or the other.

The electronics were switched off, and then switched back on again, and the Wing Sail returned to operation.

Later inspection of the Wing Sail controller revealed evidence of corrosion, despite the epoxy coating.

It appears that the epoxy coating was incomplete, and allowed salt water to come in contact with the electronic components.

It is believed that this was the reason for the failure of the Wing Sail controller.

It seems likely that the voyage 11 days earlier, where the Wing Sail would have been submerged in the sea water may have allowed salt to remain in voids or fissures in the epoxy coating.

Whilst dry, the salt would not have had any effect.

 

It seems likely that on the next voyage, the moist atmosphere would allow the salt to affect the circuitry.

Wing Sail Controller showing corrosion - top view

Wing Sail Controller showing corrosion - side view

To address this problem, the Wing Sail controller will be fully potted in epoxy.

A new PCB mount, incorporating a potting box has been 3D printed.

New Wing Sail Controller, located in 3D printed potting box

The Importance of True Wind Direction for a Sailing Drone

 The Importance of True Wind Direction for a Sailing Drone - 4th Bass Strait Voyage

An opportunity arose on the evening of 22/3/2024 to commence a passage in Bass Strait from Torquay to Western Port. An 80km (50 mile) passage that would be expected to take around 48 hours.

The mission failed within a couple of hours, with the vessel running ashore before midnight of the same evening.

Torquay Fishing Beach 7pm launch

Conditions were calm as the vessel ran downwind to the lee shore.

The vessel was found with minor damage on the beach.

The first question was whether the damage occurred at sea and caused the failure of the mission, or whether the damage occurred when the vessel was beached.

The equipment housing was completely intact and the equipment was fully operational.

Later analysis of the log files contained on the SD Card showed that the failure was due to the sailing algorithms making bad decisions.

Midnight on the beach

Spot GPS Satellite position reports

The sailing algorithm aims to get the vessel to the next waypoint, while remaining within a maximum permissible cross track error (CTE) from the rhumb line. 

It does this by trying to sail the course, using the favoured tack.

If it can't make the waypoint directly, it will sailing until the CTE reaches the Maximum CTE (boundary) permitted for the current waypoint and then change tack.

If it is unable to directly sail for the next waypoint, it is assessing at all times whether it is able to reach the next waypoint directly on the other tack.

The determination of whether it is possible to sail directly to the waypoint on the other tack is a key issue.

Analysis of the logs showed the following:

• The vessel could not quite sail directly to waypoint.
• It gradually reached CTE Max on the leeward side of the course (port side of course) on starboard tack.
• Having reached the boundary, it gybed on to port tack.
• Then it reassessed the course to the waypoint  (if it were to change starboard tack) and decided it was directly sailable.
• So, it gybed back on to starboard tack again, but it wasn't directly sailable, and it was still beyond Max CTE.
• The process repeated.
• The effect was to gybe and gybe again. effectively running downwind, until it beached on the lee shore.

Position Log from SD Card

The key issue was the wrong assessment of whether the waypoint could be directly reached on the other tack.

When racing sailing a full size yacht it can be tricky to assess when to tack to make the next mark without overlaying or underlaying.

The algorithm for assessing whether the waypoint can be reached directly on the other tack is as follows:

• Compare the Bearing to Waypoint (BTW) with the True Wind Direction (TWD) less the minimum sailing angle from the wind, less an additional sailing margin.
• The sailing margin is currently set to 25 degrees. This is effectively a measure of how much to overlay the mark before considering it is sailable.

Correctly estimating the TWD is critical for making correct decisions, and this was the source of the problem.

The determination of TWD is performed approximately as follows:

• True heading of vessel (HDG_T),
• less Apparent Wind Angle (AWA) 
• less correction adjustments to include an approximation for boat speed..

So the main requirements for determining TWD is to measure an accurate vessel heading, and apparent wind angle.

The main problem with Voyager 2.7 was the accuracy of the magnetic heading.

The following plot illustrates the relationship between the vessel heading and the calculated AWD, as it was during the failed mission.

The plot was prepared from data recorded on the SD Card, while the vessel was rotated on a turntable to assist gathering measurements.

It shows that the AWD varied by up to 47 degrees with different headings.

AWD versus Heading as it was for the mission

The compass was upgraded by removing the UFSF Max IMU , and replacing it with the LSM303.

The improvements brought by the new compass, and careful calibration, and minor adjustments to the calculations greatly improved the results.

The resulting plot of AWD versus Heading after the upgrades performed, yielded significant improvements, as seen by the near flat orange line.

The variation of AWD was less the 6 degrees with heading completing three full turns.

AWD versus Heading after upgrades

Then end result should yield a much more accurate AWD and hence estimated TWD.

The next ocean voyage should be telling.

 

Satellite Communications - Part 2

The Teensy 3.6 microcontroller has reached the end of production and the Teensy 4.1 is now the suggested device for projects with large I/O requirements. So the Voyager Controller PCB has been updated to V4.0 and then V4.1 as part of accommodating the Teensy 4.1 with connections for the additional serial port supporting the Astronode device as well as a general update.

The Teensy 4.1 is not pin-compatible with the Teensy 3.6, and also required some software changes. Once the new boards arrived, it required a few days to update, integrate and test the updated libraries, and verify their operation, and verify the new board.

Voyager Controller V4.0 - Integration Testing

The integration of the Astrocast Astronode S+ has gone well.
It now supports periodic upload of telemetry data from the vessel, as well as the ability to receive commands to amend the current mission, by inserting new waypoints in the list, or updating existing waypoints. This includes the currently active waypoint.
The aim is to be able to change the route in response to weather systems.

New Version 4.1 controller installed - ready for on water trials

Flip-top Antenna Deck to access hidden components

The satellite telemetry is currently configured to be recorded every 2 hours and uploaded to the satellite at the next available opportunity.

The satellite telemetry data that is uploaded is as follows:

            Date and Time

            Battery Voltage

            Battery Current

            Wing Sail Battery Voltage

            Average Roll angle

            COG

            COG Average 

            Wing Sail Angle

            SOG Average 

            SOG m/s

            Heading Magnetic

            BTW

            DTW

            CTE (cross track error)

            Max CTE (allowed for this waypoint)

            Deck Temp 

            Equipment Temp

            Barometric Pressure

            Wing Sail Time Since Last Response

            Mission index (way point number)

            Current Position

            Previous waypoint

            Next waypoint

Update January 2024 - bad news with the Satellite Comms:

I've been testing and integrating the Astrocast Astonode S+ devices into the Voyager hardware and software since April 2023.

I have an Astrocast account that I've been operating in trial mode to allow the integration testing to proceed before the account is activated and billing commences.

It looks like there's been a misunderstanding about the monthly fees once billing commences.
I thought it would a few dollars per month, as per their web site (which has now been updated in the last 24 hours after I pointed out the pricing I was expecting.).

The Astocast website previously displayed the following data plans shown in the screenshot below. These prices are very attractive, with no activation fee and with no hidden fee.

The data plans have been removed from one page of the Astrocast website. That's the page I cited during our discussion.
But the data plans with no activation fee and no hidden fee are still present on two other pages:

• Event Portfolio & Service - Astrocast - Taking IoT Further
• Astropreneur Program - Astrocast, Taking IoT Further

It seems that this pricing only applies once a service fee of around CHF 50/month or CHF 500/month is paid, depending on the service contract.
They have directed me to a reseller, Soracom.
I'm currently waiting on a response from Soracom. Let's see how that goes...