[neilh20]

Congrats – nice to see.

I guess the fundamental problem of power management is still there and can bite any time the power drops – and then likely will require a site visit to get the site restarted. Getting enough power is a band aid to enable reporting readings. So long as the power supplied exceeds the power used by a good margin it works. 🙂 I’ve found the real fix is in software using higher voltage thresholds – but it causes tradeoff

Its interesting the shape of the 2nd charging pulse – indicates lost data.  Stepping to the larger graph, its more visible.

So the next layer of reliability issues, with the fix being in software. I predicted the problem in 2018, and had a solution for it in 2020 that I’m using,

Orderly data delivery – Queue messages for later sending,

So just advertising there is a fix for it 🙂

 

 

Attachments:

1. 

   240223_1121-SpokaneR-SpkaneValley_Depth.png
   

[neilh20]

@jamesp  thanks for sharing. Interesting where the “banana skins” are to slip on.

Nice to know its got low quiescent current – end of last year I also  got the same  SanDisk Industrial MLC MicroSD SDHC UHS-I Class 10 SDSDQAF3-008G-I withs its specified Operating Temp Range: -25°C to 85°C , but haven’t measured its current 🙂

I’ve bought uSD with “white” so I can write on it to track individual cards.

I was using SanDisk Ultra SDSQUNS-016G-GN3MN 16GB (10 Pack)  for most of my systems – however it hasn’t got an outdoor temperature specification. I had a problem with some occasional resets on one of my systems, so I switched it to the above.

[neilh20]

@jamesp I wonder how the systems are faring?

[neilh20]

A follow up – I had this running for about 6 weeks last year.

The code is in my fork is at  https://github.com/neilh10/ModularSensors/tree/release1/examples/samd_log_display_mmw  and is available to the main at any time.

For test purposes, I positioned the temperature sensors in my office, attached to different areas round the heating vent, with the winter heating kicking in.

This is is very much an alpha test – its designed to do an end-2-end system test and characterize areas that need more work.

What showed up was the difficulty in labeling or easily “tracking” what was being measured . It turns out there isn’t the capability to give multiple sensors of the same type (DS18 thermistor) a “nickname” to describe the location that they are positioned in

I documented it here https://github.com/ODM2/ODM2DataSharingPortal/issues/686

For the DS18 temperature sensor readings to be easier to decode and track – the Arduino world of make it easy – I would want to add distinguishing labels – hints to understand the graphs –  “Vent Output”, “Vent Underneath”, “Vent-Horiz 1ft”,  “Window ledge”

Of course for analysis it can be repeatedly downloaded – brought into a spreadsheet, and meaningful labels added/

 

Some other issues,

I wanted to configure the DS18 temperature sensor s/n in my .ini file – to make it easier to manage the sensors. However for some reason moving the DS18 from standard static initialization, to a dynamic initialization and then setting the s/n number failed to work. It should be easy – but that’s the story with software – I’m missing something somewhere. In the interests of trying out the alpha test config  I created a bug to be able to track it later and moved on

As I had discovered in testing, the Wio Terminal internal WiFi networking module RTL8720 – has some problems with hanging when dealing with external conditions. A couple of other people have the same issue, No answer from Wio Terminal about this. https://forum.seeedstudio.com/t/rpcwifi-problem-when-wifi-is-lost/262273/4

As expected from a circuit analysis the Wio Terminal, while nice looking, doesn’t do Low Power. I had wound down the clock frequency to use as little power as possible, and still the 6Ahr battery  lasted about 2.5days.

Conclusion : This has shown that the  ModularSensors modifications for SAMD51/SAME51 processors with ARM tools works. The SAMD51/SAME51 is good fit for a sensor node as it has an architecture with number of configurable serial peripherals – including  possibly up to 6 serial UARTs+I2C+SPI,  and 32DMA and 2MBytes program flash + low cost external flash that can be dual usage.  However I need to try with some other SAMD51 boards that can achieve a low power state. I have a few Adafruit options, some  with LCDs.  I also need to be able to easily interface to an LTE Modem,  and for Modbus generate +12V & RS485.

Other processors include the newish lowish power, ESP32-C6 with 3 UARTS could also be attractive, but is a different tool set. The ESP32 based “M5 Stack” boards are trying for lowpower by turning off the electronics  all together.

[neilh20]

I’m very much a fan of ModularSensors and the discussion board as a place to be real with technical issues.   I do so appreciate the unique place EnviroDIY has been curated to be, to try some remote wireless terrestrial monitoring. Its the nitty gritty of environmental monitoring and the engineering it takes.   Its tough to find out what these services cost – so hope it works out.  I’ve contacted the funder of the project I’ve been in to let them know of the late changing conditions. On a commercial scale the initial entry is pretty low cost , but going to be expensive if  it scales.  I hope it  can keep up with the demanding changing technical bar.

There are some really nice features mentioned, though I’m curious as to whether this constitutes a “Service Level Agreement”, and is there a verification plan.

A critical technical statement  that is missing is the expected response from the server. Device POSTing is only as good as the servers response time.

There have been a couple of windowed losses in data storage on MMW and the subject doesn’t seem to be addressed-

https://github.com/ODM2/ODM2DataSharingPortal/issues/605 

https://github.com/ODM2/ODM2DataSharingPortal/issues/685

and I brought it up locally –  https://www.envirodiy.org/topic/mmw-lost-readings/

so curious how real-world issues are going to be addressed.  Many thanks. Neil

[neilh20]

I haven’t had much luck getting to 0.5mA with mega1284, it was with the Mayfly 0.5b- https://www.envirodiy.org/topic/testing-power-consumption/#post-15252   My notes on measurements is that it was between 2-2.5mA.  I did get well beneath 0.5mA   on another project with mega2560, but that was a lot of hardwork, and also ended up with a higher value of mA for the production version – as the modem connection really consumes the energy.

In looking at testing the whole system, including production software – I came to the view its the apparent reliability of the overall system – and managing the power demand, and delivering readings to MMW, that results  in a stable overall system – requiring less intervention and maintenance.

I put a model together for power used over a day with the most active subsystems. I enclose “Mayfly MS Energy Useage 231218.xlsx”

This is really paying off with this system https://monitormywatershed.org/sites/TUCA_GV01/

Attachments:

1. Mayfly-MS-Energy-Useage-231218.xlsx

[neilh20]

@jp – the charger relationship doesn’t quite work that way. The 5V 2.0A is typically the stated maximum the charger can supply – however for the battery charger circuit there are limitations. I use a USB Meter to determine what is actually going on.

The Mayfly 1.x charger circuit is to 0.5A as default  but can be changed to 1.0A. I always change my boards to 1.0A  See ” SJ15″ https://www.envirodiy.org/mayfly/hardware/jumper-settings/

The yellow “Charge” LED at location “S” only comes on when it is charging the battery – so a sign of  it charging is that it comes on when plugging in, and then when completed it goes off.  These are different than the LEDs at USB-C  location “C” labeled “power” and “USB”

The LiIon charger does manage the battery and has effective feedback

For LiIon power storage calculations its estimating Power-Out over perhaps two weeks, the length of no solar say in a storm – its a soft figure but that’s my estimate.   Then when solar returns – the power-solar that is going to recharge the battery – in says two days of solar in winter with a cold battery.

From what I saw your solar panel was providing a charge earlier in the year, but now with winter conditions is not.

Power-Out is also dependent on many factors, the largest power user is usually the modem – and this can vary by time of year – I find it takes 26secs to connect to the tower, and then it takes time to establish a TCP/IP link to the MonitorMyWatershed  – and then POST the readings, and sometimes even longer for a timeout.  If the physical location is on the edge of the cell, then connection conditions can really way out.  In addition if the Li-Ion battery is at a low charge state, and also cold, it sometimes can’t supply all the power needed to connect the signal and effectively reset the process.  The Li-Ion battery power availability might be on the edge temperature wise, work in the afternoon when it has warmed and then fail when its colder.

For battery voltages – my measurements on the Vbat indicate its non-linear. Its not very accurate particularly in the range that is really needed 3.4V-3.8V – I’ve calculated that the error in reading using Vbat is +/-0.2V .

I use a 3.8V threshold to gate using the modem. That is when the Vbat measures 3.8V – the battery could be somewhere 3.6 or 4.0V and I don’t invoke the modem.  I use a value of 3.6V to stop doing any processing.

The SpokaneRiver-Peaceful.ino and SpokaneR-SpokaneValley.ino both use a 3.55V to gate the modem – which are defined in ModularSensors\examples\DRWI_SIM7080LTE\DRWI_SIM7080LTE.ino

However one systems battery goes low – and it resets on power demand, and the other systems battery stays above 4.0V

The way I read ModularSensors\examples\DRWI_SIM7080LTE\DRWI_SIM7080LTE.ino  is its an “example” – there is no testing of edge conditions that I have seen of that actually test and verify that a SIM7080 modem can work at 3.55V in cold weather.

I have actively monitored and tested 4.4Ahr battery and documented it  here https://github.com/EnviroDIY/EnviroDIY_Mayfly_Logger/issues/32#issuecomment-959807618 .

Due to high value resistor ratio its also a noisey signal – and I document it here  – https://github.com/EnviroDIY/EnviroDIY_Mayfly_Logger/issues/36 

IMHO the measurement technique of reading  the physical ADC  twice in succession and using those values separately isn’t good for this type of high impedance noisey ADC measurement. I use an algorithm to read the ADC a number of and then take the lowest reading for those Vbat readings, and then use it in both decision loops.

Power availability is complicated when its on the edge. As far as I can see the Mayfly 1.x  Vbat has never had its accuracy defined. Accuracy is related to both resistor variations, resistor noise and ADC Vref accuracy.  Mayfly 1.x Vref is a vast improvement on the  Mayfly 0.5b hardware realization.

So this is what I would think is a good starting point

//Get realistic battery voltage float vBatLow = getLowestBatteryVoltage(); #define VBAT_WORKING_THRESHOLD 3.6 #define VBAT_MODEM_THRESHOLD 3.8 // At very low battery, just go back to sleep if (vBatLow < BAT_WORKING_THRESHOLD) { dataLogger.systemSleep(); } // At moderate voltage, log data but don't send it over the modem else if (vBatLow < VBAT_MODEM_THRESHOLD) { dataLogger.logData(); } // If the battery is good, send the data to the world else { dataLogger.logDataAndPublish(); }

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

//Get realistic battery voltage float vBatLow = getLowestBatteryVoltage(); #define VBAT_WORKING_THRESHOLD 3.6 #define VBAT_MODEM_THRESHOLD 3.8   // At very low battery, just go back to sleep if (vBatLow < BAT_WORKING_THRESHOLD) { dataLogger.systemSleep(); } // At moderate voltage, log data but don't send it over the modem else if (vBatLow < VBAT_MODEM_THRESHOLD) { dataLogger.logData(); } // If the battery is good, send the data to the world else { dataLogger.logDataAndPublish(); }

 

The simplest solution for the current systems is provide enough power to keep the LiIon battery charged up, with a bigger solar panel.  If  at this time of year if it starts to drop below a measured 3.9V go out and plug in a USB Charge bank to boost the battery.

Alternatively if easy also modify the Mayfly 1.x for SJ15 so that it takes more charge when available.

Once there is more solar it probably will maintain more power in the battery.

 

 

[neilh20]

Matt Thanks for posting. I’m away from my computer till next week, so I’ll be interested to look when I get back.
Seems like SpokaneRiver-Peaceful has good solar, and the battery never gets low.

[neilh20]

@oakleynode  wow  interesting you are looking at RSSI and it has a lot of variation. I haven’t characterized the RSSI result for a high percentage of failing calls.

Another system that is also on the edge, and I added a new antenna that improved it,  the reading is  -81dBm  https://monitormywatershed.org/sites/TUCA_Mi06/

The modem is a  Digi CAT-M1  XB3-C-A2 with Verizon service. I haven’t got a real figure of ranges that should work

A test system is giving a crazy -1dBm – https://monitormywatershed.org/tsv/tu_rc_test07/4904/  – maybe its too strong to characterize!

The retrieved RSSI is the measurement of the last call, not the current one. Not sure what it indicates if the call fails completely – doesn’t go through.   for GV01 its attempting to make calls every 15minutes, and then subsequently 15minutes later would be retrieving the  RSSI.

For Mi06 it only makes a call every hour, though it retrieves the RSS reading on every 15minutes, to its likely the RSSI is the same value between calls.

For cell reflection off the ocean – you could look it up in a text book – but with waves maybe too chaotic. Not sure what you do with any answer, as it works or it doesn’t. It might be an interesting answer if you are wanting to compare against different frequencies – say would LoRa 900MHz be more reliable, but requires data packing. I haven’t figured out to easily do LoRa with MMW.

I keep an internal log on the uSD of connection attempts, and time to make the connection. Mostly that’s been useful for validating that connections where happening, that is verifying the device software.  It also allowed me to show that  the server was loosing data and in early 2023 the server response time was going out .

[neilh20]

@oakleynode something to note is that the MMW/ODM2 packet design with UUIDs make an extra heavy packet. The packet design is not optimally designed for wireless conditions, the thingspeak packets are much lighter. If your site is on the edge of the wireless range,  which varies a lot of with weather conditions, its going to have more trouble with the MMW UUID overheads. A site that I have on the very edge of wireless connectivity is https://monitormywatershed.org/sites/TUCA_GV01/.  I implement a sequence number in all my packets to be able to easily characterize reliability. I have been upgrading the antenna on the site to be able to improve the wireless signal strength.

I’ve found  the server is pretty exacting, and the error responses are wip. In addition due to some undocumented compression routines on the server, if the POST timestamp is too old, and that region of the database its going to insert into is compressed,  it then initiates an uncompres for that region and that can take an undefined  looong, time . Which can the client POSTer to define a times-out.

 

I’m an advocate of reliability for ModularSensors,   – and that also has meant characterizing the server, and in the process of testing my devices I’m finding server issues, one example   https://github.com/ODM2/ODM2DataSharingPortal/issues/685

My reliability enhancements  can do multiples POSTs per connection . Over a celluar connection (with 9600bard to the modem) the first POST has had responses of ~ 2.5sec, and subsequent ones 10seconds. However the server is not set up to take too many serial POSTs.  Discussion https://github.com/ODM2/ODM2DataSharingPortal/issues/673#issuecomment-1692514992

For the ODM example,  seems to me it is missing the “Content-Length: ”

The best documentation is the code.

https://github.com/EnviroDIY/ModularSensors/blob/master/src/publishers/EnviroDIYPublisher.cpp

I got the POST working on a Wio Terminal device (WiFi), but I needed to get wire-shark out to be able to sniff the packet to see exactly what the WioT device driver was outputting.

Here is an example of a trace from the transmitting device that succeeded (with a couple of changes to UUIDs so this won’t work)

POST /api/data-stream/ HTTP/1.1
Host: data.envirodiy.org
TOKEN: d9e65186-6cf6-4803-xxxx-4d9b1142bd48
Content-Length: 508
Content-Type: application/json

{“sampling_feature”:”851d0f00-3d8b-41c7-xxxx-40cb86b9ad2c”,”timestamp”:”2023-10-27T16:04:49-08:00″,”740cc4f8-6bc5-471a-811c-e0977f286f51″:3,”1f239d16-e6e4-4f76-ae98-0b274c75c79e”:48.04,”5ab3af01-bf49-4187-893d-276d7d256cd7″:17.93,”fd4e9448-8f67-4b5b-bd73-630b0659af8b”:17.1875,”586f9006-261b-4c5a-8abe-c93d5ec799fe”:17.8750,”87e166b4-0810-424f-bcd1-bbfaf9bda24d”:17.5625,”f10be729-b2d0-48ba-9c30-912274985b83″:17.7500,”27d7e570-7ef1-4ab5-af94-dfef2eeb19c1″:4.101,”66b0642a-d8c9-4e25-9a8f-39e081c188fc”:6.557}

and in this case over WiFi the response is faster

— Response Code — 201 waited 632 mS Timeout 10000

regards

[Author]

Posts