Welcome to EnviroDIY, a community for do-it-yourself environmental science and monitoring. EnviroDIY is part of WikiWatershed, a web toolkit designed to help citizens, conservation practitioners, municipal decision-makers, researchers, educators, and students advance knowledge and stewardship of fresh water. New to EnviroDIY? Start here

Parameter Threshold Notifications

Home Forums Mayfly Data Logger Parameter Threshold Notifications

Tagged: 

Viewing 13 reply threads
  • Author
    Posts
    • #12908
      lemon
      Participant

      Has anyone set up notifications when Mayfly logger stations data exceeds a prescribed threshold? For example, if conductivity were to exceed 500 us/cm an email would be sent to a designated email address.

    • #12910
      Adam Gold
      Participant

      You can do this remotely if you send data to ThingSpeak (already supported by ModularSensors). There are caps on how many ThingSpeak channels you can have and still have it be free (4 per account, I think), but you can make a ThingSpeak channel email or tweet at you when it meets certain conditions. Here’s a similar question about alarms from the Mayfly: https://www.envirodiy.org/topic/programming-mayfly-to-send-alerts/

      Emailing from ThingSpeak involves using an IFTTT applet and some built-in ThingSpeak apps: https://www.mathworks.com/help/thingspeak/analyze-channel-data-to-send-email-notification-from-ifttt.html

      Tweeting functionality is built in to ThingSpeak: https://thingspeak.com/apps

    • #12911
      lemon
      Participant

      Thanks Adam!

    • #13358
      neilh
      Participant

      To piggy back on this discussion – I’m wondering if anybody (@srgdamiano?) knows if there is a way of doing a POST/GET to request data from the ODM database for the latest row, or be able to access latest readings via WaterML.
      The readings POST structure is described here
      https://github.com/ODM2/ODM2DataSharingPortal/blob/master/doc/example_rest_requests.md

      https://www.envirodiy.org/midstream-making-real-time-data-publicly-available/ from early architecture suggest access through WOFpy

      What I’m looking for is to be able setup elsewhere a periodic request on the latest reading to be able to do a threshold check.

      I know I can request data from ThingSpeak in this fashion, just wondering if there is a spec for it for data.enviroDIY.org
      many thanks

    • #13361
      Sara Damiano
      Moderator

      The WoFpy endpoint is here: http://data.wikiwatershed.org/wofpy/

    • #13363
      Sara Damiano
      Moderator

      Just a note on sending to ThingSpeak: there’s no free tier for anything but “personal use.” If you set your MathWorks/MatLab/ThingSpeak account up as anything else (even non-profit/educational) you only get 30 days free and have to pay beyond that, no matter how low your usage is.

    • #13365
      neilh
      Participant

      Gosh @srgdamiano wonderful!. ..

      As every another question … (thankyou thankyou in advance)

      So following the examples and trying for some GetValues on EnviroDIY_Mayfly_Temp from
      https://data.envirodiy.org/sites/LCC45a3/
      with this
      http://data.wikiwatershed.org/wofpy/rest/1_1/GetValues?location=envirodiy:LCC45a3&variable=envirodiy:EnviroDIY_Mayfly_Temp&startDate=2019-09-01T00:00:00&endDate=2019-11-01T02:30:00

      I get a response but no <values> that I would think I should get.
      Any pointers – many thanks.

    • #13366
      Sara Damiano
      Moderator

      There are some bugs with WOFpy right now; I’d thought some had been fixed, but it looks like the only-returning-one-point issue is known and un-fixed: https://github.com/ODM2/ODM2DataSharingPortal/issues/432

      I very rarely look at the WOFpy endpoint, so I’m not always up-to-date on it’s working status. Sorry!

    • #13367
      neilh
      Participant

      OK I think I’ve figured out something. I can only get the last variable at the moment (432) .. which is what I need for my real analysis of looking at the last value of the voltage returned and and checking it it has dropped below a threshold.

      http://data.envirodiy.org/sites/TU-RC-01/ visually its 16.0V the first Battery Voltage “All_ExternalVoltage_Battery” and I can also see with a download of series=1906

      So then using
      http://data.wikiwatershed.org/wofpy/rest/1_1/GetValues?location=envirodiy:TU-RC-01&variable=envirodiy:All_ExternalVoltage_Battery
      comes back with something that includes

      <values>
      <value methodCode=”2″ qualityControlLevelCode=”1″ censorCode=”nc” dateTime=”2019-11-18T19:27:02″ dateTimeUTC=”2019-11-19T03:27:02″ sourceCode=”611″ timeOffset=”-08:00″>16.0</value>

      which seems like I read as

      <values>
      <value …>16.0</value>

      and its the 16.0 Volts that I’m after. So I need to parse to find that and compare it to my threshold.

    • #13376
      neilh
      Participant

      On the ThingSpeak thanks for pointing it out – they have become painful. I have a number of legacy connections I’m setup up for.

      Not knowing the architecture of the ODM2/WOF and WaterML 1.1 it was more of a reference.
      Thingspeak used to be a lot more open, but Mathwork$ acquired them, and I guess they are making money out of it.
      Which is to be expected, but I so appreciate enviroDIY coming along for environmental monitoring (and thanks to the NSF and William Penn Foundation funders)

      I am digging into Python libs in parsing the WaterML – but if you have any examples on getting to the data that would be great.

    • #13382
      Sara Damiano
      Moderator

      Have you found the ulmo library: https://github.com/ulmo-dev/ulmo It can parse the WaterML.

      I believe it’s what Model My Watersehd and other tools we have are using. I haven’t actually used it myself, so I can’t help with it more than giving you the link.

    • #13386
      neilh
      Participant

      Thanks for ulmo reference – I can sort of see something there, but its got a lot of layers and I can see it understands the response from a http://data.wikiwatershed.org/wofpy/rest/1_1/GetValues?
      of the form
      <timeSeriesResponse xmlns=”http://www.cuahsi.org/waterML/1.1/&#8221; xmlns:ns0=”http://www.cuahsi.org/his/1.1/ws/”&gt;

      Well in the spirit of try many things, be prepared to fail, I’m putting aside trying to understand the namespaces built in here defined by
      xmlns=”http://www.cuahsi.org/waterML/1.1/&#8221; xmlns:ns0=”http://www.cuahsi.org/his/1.1/ws/&#8221;

      So just an FYI if there are an XML gurus, I’m trying to retrieve the simple value, and whenever I try to search down using xml.etree.ElementTree I can’t null out the effects of xmlns and xmlns:ns0.

      A sensor value is encoded as “timeSeries” and shows

    • #13387
      Sara Damiano
      Moderator

      I don’t think you should have to be messing with eTree if you’re using ulmo. There’s a get_values function that returns a simple python dictionary: https://ulmo.readthedocs.io/en/latest/api.html#ulmo.cuahsi.wof.get_values The WSDL URL is https://monitormywatershed.org/wofpy/soap/cuahsi_1_1/.wsdl.

      I hate having to use the eTree library. I find trying to navigate through layers of XML to be approximately 0% fun.

    • #13390
      neilh
      Participant

      Thanks for the front door 🙂 eTree was going down a whirly pooll!

      This worked for me, though maybe there is a nicer way.
      The output is value_num – which can be used for the threshold check.

      The retrieved ‘sites_values_dict’ is

      The debug out looks like this

Viewing 13 reply threads
  • You must be logged in to reply to this topic.