Welcome to EnviroDIY, a community for do-it-yourself environmental science and monitoring. EnviroDIY is part of WikiWatershed, a web toolkit designed to help citizens, conservation practitioners, municipal decision-makers, researchers, educators, and students advance knowledge and stewardship of fresh water. New to EnviroDIY? Start here

Parameter Threshold Notifications

Home Forums Mayfly Data Logger Parameter Threshold Notifications

Tagged: 

Viewing 13 reply threads
  • Author
    Posts
    • #12908
      Jake Lemon
      Participant

        Has anyone set up notifications when Mayfly logger stations data exceeds a prescribed threshold? For example, if conductivity were to exceed 500 us/cm an email would be sent to a designated email address.

      • #12910
        Adam Gold
        Participant

          You can do this remotely if you send data to ThingSpeak (already supported by ModularSensors). There are caps on how many ThingSpeak channels you can have and still have it be free (4 per account, I think), but you can make a ThingSpeak channel email or tweet at you when it meets certain conditions. Here’s a similar question about alarms from the Mayfly: https://www.envirodiy.org/topic/programming-mayfly-to-send-alerts/

          Emailing from ThingSpeak involves using an IFTTT applet and some built-in ThingSpeak apps: https://www.mathworks.com/help/thingspeak/analyze-channel-data-to-send-email-notification-from-ifttt.html

          Tweeting functionality is built in to ThingSpeak: https://thingspeak.com/apps

        • #12911
          Jake Lemon
          Participant

            Thanks Adam!

          • #13358
            neilh20
            Participant

              To piggy back on this discussion – I’m wondering if anybody (@srgdamiano?) knows if there is a way of doing a POST/GET to request data from the ODM database for the latest row, or be able to access latest readings via WaterML.
              The readings POST structure is described here
              https://github.com/ODM2/ODM2DataSharingPortal/blob/master/doc/example_rest_requests.md

              https://www.envirodiy.org/midstream-making-real-time-data-publicly-available/ from early architecture suggest access through WOFpy

              What I’m looking for is to be able setup elsewhere a periodic request on the latest reading to be able to do a threshold check.

              I know I can request data from ThingSpeak in this fashion, just wondering if there is a spec for it for data.enviroDIY.org
              many thanks

            • #13361
              Sara Damiano
              Moderator

                The WoFpy endpoint is here: http://data.wikiwatershed.org/wofpy/

              • #13363
                Sara Damiano
                Moderator

                  Just a note on sending to ThingSpeak: there’s no free tier for anything but “personal use.” If you set your MathWorks/MatLab/ThingSpeak account up as anything else (even non-profit/educational) you only get 30 days free and have to pay beyond that, no matter how low your usage is.

                • #13365
                  neilh20
                  Participant

                    Gosh @srgdamiano wonderful!. ..

                    As every another question … (thankyou thankyou in advance)

                    So following the examples and trying for some GetValues on EnviroDIY_Mayfly_Temp from
                    https://data.envirodiy.org/sites/LCC45a3/
                    with this
                    http://data.wikiwatershed.org/wofpy/rest/1_1/GetValues?location=envirodiy:LCC45a3&variable=envirodiy:EnviroDIY_Mayfly_Temp&startDate=2019-09-01T00:00:00&endDate=2019-11-01T02:30:00

                    I get a response but no <values> that I would think I should get.
                    Any pointers – many thanks.

                  • #13366
                    Sara Damiano
                    Moderator

                      There are some bugs with WOFpy right now; I’d thought some had been fixed, but it looks like the only-returning-one-point issue is known and un-fixed: https://github.com/ODM2/ODM2DataSharingPortal/issues/432

                      I very rarely look at the WOFpy endpoint, so I’m not always up-to-date on it’s working status. Sorry!

                    • #13367
                      neilh20
                      Participant

                        OK I think I’ve figured out something. I can only get the last variable at the moment (432) .. which is what I need for my real analysis of looking at the last value of the voltage returned and and checking it it has dropped below a threshold.

                        http://data.envirodiy.org/sites/TU-RC-01/ visually its 16.0V the first Battery Voltage “All_ExternalVoltage_Battery” and I can also see with a download of series=1906

                        So then using
                        http://data.wikiwatershed.org/wofpy/rest/1_1/GetValues?location=envirodiy:TU-RC-01&variable=envirodiy:All_ExternalVoltage_Battery
                        comes back with something that includes

                        <values>
                        <value methodCode=”2″ qualityControlLevelCode=”1″ censorCode=”nc” dateTime=”2019-11-18T19:27:02″ dateTimeUTC=”2019-11-19T03:27:02″ sourceCode=”611″ timeOffset=”-08:00″>16.0</value>

                        which seems like I read as

                        <values>
                        <value …>16.0</value>

                        and its the 16.0 Volts that I’m after. So I need to parse to find that and compare it to my threshold.

                      • #13376
                        neilh20
                        Participant

                          On the ThingSpeak thanks for pointing it out – they have become painful. I have a number of legacy connections I’m setup up for.

                          Not knowing the architecture of the ODM2/WOF and WaterML 1.1 it was more of a reference.
                          Thingspeak used to be a lot more open, but Mathwork$ acquired them, and I guess they are making money out of it.
                          Which is to be expected, but I so appreciate enviroDIY coming along for environmental monitoring (and thanks to the NSF and William Penn Foundation funders)

                          I am digging into Python libs in parsing the WaterML – but if you have any examples on getting to the data that would be great.

                        • #13382
                          Sara Damiano
                          Moderator

                            Have you found the ulmo library: https://github.com/ulmo-dev/ulmo It can parse the WaterML.

                            I believe it’s what Model My Watersehd and other tools we have are using. I haven’t actually used it myself, so I can’t help with it more than giving you the link.

                          • #13386
                            neilh20
                            Participant

                              Thanks for ulmo reference – I can sort of see something there, but its got a lot of layers and I can see it understands the response from a http://data.wikiwatershed.org/wofpy/rest/1_1/GetValues?
                              of the form
                              <timeSeriesResponse xmlns=”http://www.cuahsi.org/waterML/1.1/&#8221; xmlns:ns0=”http://www.cuahsi.org/his/1.1/ws/”&gt;

                              Well in the spirit of try many things, be prepared to fail, I’m putting aside trying to understand the namespaces built in here defined by
                              xmlns=”http://www.cuahsi.org/waterML/1.1/&#8221; xmlns:ns0=”http://www.cuahsi.org/his/1.1/ws/&#8221;

                              So just an FYI if there are an XML gurus, I’m trying to retrieve the simple value, and whenever I try to search down using xml.etree.ElementTree I can’t null out the effects of xmlns and xmlns:ns0.

                              A sensor value is encoded as “timeSeries” and shows

                            • #13387
                              Sara Damiano
                              Moderator

                                I don’t think you should have to be messing with eTree if you’re using ulmo. There’s a get_values function that returns a simple python dictionary: https://ulmo.readthedocs.io/en/latest/api.html#ulmo.cuahsi.wof.get_values The WSDL URL is https://monitormywatershed.org/wofpy/soap/cuahsi_1_1/.wsdl.

                                I hate having to use the eTree library. I find trying to navigate through layers of XML to be approximately 0% fun.

                              • #13390
                                neilh20
                                Participant

                                  Thanks for the front door 🙂 eTree was going down a whirly pooll!

                                  This worked for me, though maybe there is a nicer way.
                                  The output is value_num – which can be used for the threshold check.

                                  The retrieved ‘sites_values_dict’ is

                                  The debug out looks like this

                              Viewing 13 reply threads
                              • You must be logged in to reply to this topic.