Reply To: Response code 504 from data.envirodiy.org

Hi Matt
You’re doing some good characterization work. Thankyou for sharing it.
A basic issue to keep in mind is that all communications can be unreliable as there is complex telecoms infrastructure.

An arbitrary wireless placement’s reliability is difficult to quantify ~ distance to cell head, direction of wind, fog/rain etc. A wireless network location needs to be categorized for being reliable.

I use the office based WiFi to attempt to have a reliable reference point. (and I’m still seeing timeouts 504 with data delivery and also some posts not being inserted)

From an industry context, communication protocol retry’s are the only way to provide for reliable data delivery, and I raised this feature request.
https://github.com/EnviroDIY/ModularSensors/issues/194

I haven’t managed to implement it in my EnviroDIY systems yet either https://github.com/neilh10/ModularSensors, but its high on my list, and periodically I give me a kick on the butt for not having got it done. Like now. Ideally this type of reliable delivery is implemented early so that overall reliability can be characterized. I implemented another system, and even though networks in rural areas could be down for weeks, the data would eventually ALL be pushed to the server.

I have implemented a Sequence number (EnviroDIY_Mayfly_SampleNum) which increments on the Mayfly side, and then on the monitorMyWatershed it can be viewed, and any missing numbers detected.

So for a field system the sequence number appears to be continuous from Oct/24/2019, https://monitormywatershed.org/tsa/?sitecode=TU-RC-01&variablecode=EnviroDIY_Mayfly_SampleNum&view=visualization&plot=true
So for this system, running over I think an ATT CAT-M1 modem, there are only occasional packet losses of 1 – see graph generated from .xls file

That said I have been seeing a lot of (504) from my desk workstations – and haven’t dug into why.