Welcome to EnviroDIY, a community for do-it-yourself environmental science and monitoring. EnviroDIY is part of WikiWatershed, an initiative of Stroud Water Research Center designed to help people advance knowledge and stewardship of fresh water.
New to EnviroDIY? Start here

Connecting SIM7080 to New Thingspeak

Home Forums Mayfly Data Logger Connecting SIM7080 to New Thingspeak

Viewing 7 reply threads
  • Author
    Posts
    • #16225
      Zeke Holloman
      Participant

        @neilh20 @shicks @vogelrnws

        Hi there! I recently received an email from Thingspeak stating that they are planning on shutting down their legacy Thingspeak server this month. I’m working to convert my Thingspeak channels over to from mqtt.thingspeak.com to mqtt3.thingspeak.com.

        I’m using a Mayfly v1.0 alongside you all’s SIM7080 card. It publishs data to legacy ThingSpeak with no problems! I figured it would be just as simple as changing the website URL to “mqtt3.thingspeak.com”, but it’s proving more difficult than that. Have you all had any experience with this? Any suggestions are appreciated.

        Thank you and have a good one!

      • #16226
        Rick Vogel
        Participant

          Can you provide more details especially in reference to your code and how you are connecting to thingspeak?

        • #16229
          Sara Damiano
          Moderator

            Sorry, I coded up the ThingSpeak publisher ages ago only to prove to myself that I could get an MQTT publisher working. I’ve never used it beyond that. I don’t have a paid account so after the trial period I was locked out and I’m not willing to do much testing on an account tied to my personal email.

            Anyway, the code for ThingSpeak is in the ThingSpeak publisher in the publishers folder. It’s using PubSubClient under the hood. The outgoing message is formatted based on whatever the required structure was at the time. If that’s changed, then the whole formatting step will need to be rewritten.

            I wouldn’t guess the restructuring of the json or whatever ThingSpeak wants would be too hard, but I don’t have time to look at it right now. If ThingSpeak now requires a secure connection (SSL, etc) or if they more want more metadata then there might need to be more tweaking for that.

          • #16237
            neilh20
            Participant

              @zeke-holloman I agree with @srgdamiano,  Mathworks have gone down the path of making it difficult to test a small number of devices against thingspeak, since it was added to ModularSensors publishers.

              A bit of shame since I’ve had some simple devices working reliably against it since 2013.

              Also unfortunately, “the real world” is that testing is against a matrix – release ModularSensors xx.xx against thingspeak – so I think you are on the leading edge for ModularSensors Publisher v mqtt3.thingspeak  🙂

              Taking a scan of their  mqtt3,  it seems like they are suggesting you create new devices when moving to their mqtt3  – so at a GUESS, that suggests they have a whole new server architecture and subscription management. So be great if you share results – stuff that doesn’t work is as valuable as stuff that does work.

              When looking at a new service, its also useful to consider what it takes to prove how well it works  and how far you want to go.

              Typically new development goes in stages a) protoyping b) integration testing c) reliability/stability testing.

              MMW has a lot of capability, and very mature,  but in the “reliability testing” of  Mayfly 0.5 and Digi LTE CAT-M1 modems, from a debug analysis, there is a possible timing  problem with the tinyGSM device driver  https://github.com/EnviroDIY/ModularSensors/issues/396 – which suggests that if the MMW servers response where a lot faster, faster than the current fastest 1.2seconds, it might break the current deployments with Digi LTE-CAT-M1 XB3-C-A2-UT-001.   Its what happens in testing sometimes.

              I’ve had very good experience so far with “reliability/stability testing” of ModularSensors 0.32.2/(Mayfly 0.5+ Digi LTE-CAT-M1), but the server response through the cellphone network is pretty slow,  at its fastest 1.2seconds.

              For mqtt3.thingspeak you are back to “a)prototyping”.

              For  another wireless protocol LoRa/Thingsnetwork,  with max payload of 51bytes, which seems unlikely that it could work with MMW API, I’m checking out Arduino.cc mqtt/IoT or Adafruit mqtt as having plans  for smaller devices.  So I would describe I’m trying to figure how to prototype that interface.

              I should note – https://github.com/EnviroDIY/ModularSensors/releases/tag/v0.32.2 doesn’t identify what features work with new Mayfly 1.0A3 boards. The modem SIM7080 , isn’t identified as being supported yet, or tested with ModularSensors – though as its open source we can all see what’s in the software. I guess, user test at their own risk.

              I have only done minimal testing on Mayfly 1.0A3, as its typical for organizations to announce what they support and regression testing. My testing, is with a student capstone project as a standalone sensor so far, though they should get to cellular LTE integration soon.  On paper analysis I’ve identified a potential comms issues with Mayfly1.0A3  – see https://github.com/EnviroDIY/EnviroDIY_Mayfly_Logger/issues

              I’m making ModularSensors detect in runtime whether its Mayfly 0.5 or 1.0 – so the user/program doesn’t have to compile for one or the other.

            • #16244
              Zeke Holloman
              Participant

                @neilh20 @shicks @vogelrnws

                Hi all! Thank you for all of your feedback and tips! I spent some time this week looking at the mqtt3.thingspeak.com server and I was able to get a channel up and running on it.

                Overall, it’s not drastically different than mqtt.thingspeak.com. The main aspect being removed in the legacy Thingspeak server is the API keys. Thingspeak now relies on the Channel ID, the client ID, the username, and the password to send data to a specific channel.

                There are also a few formatting changes for sending the data, which can be found on this site, under July 2021:

                https://www.mathworks.com/help/thingspeak/release-notes.html#:~:text=July%25202021-,MQTT3%2520Interface,-A%2520new%2520ThingSpeak

                I did end up editing the ThingSpeakPublisher library in order to get the data to upload correctly, but I don’t think I had to change the data type.

                Here’s the Arduino code:

                 

              • #16245
                Zeke Holloman
                Participant

                  ^^ Here’s the edited cpp for the publisher

                  ^^ Here’s the edited code for the header file.

                • #16247
                  neilh20
                  Participant

                    @zeke-holloman  hey congrats,!!! Thanks for the posting and analysis, I could also build from that.

                    Wow… the power of open source that you can dive in and tweak it, so you can make it work. Go @zeke-holloman   !!!

                    Just an FYI, what I do to facilitate easy maintenance, and easy future merges, I rename my customizations to a “unique name space”  eg ThingSpeakPublisher3a.h/.cpp and then also the classes ThingSpeakPublisher3a.

                    Then if you come to do an update “git pull” and there are updates in the main line “master” ThinkSpeakPublisher.h/cpp you’ll have a smoother ride  with being able to track the updates,  and either adopt them or stay with your customizations.

                  • #16252
                    Zeke Holloman
                    Participant

                      @neilh20 Hey, thanks! WOW! That’s awesome it worked for you as well. Hope it ends up being helpful!

                      I will say, it wasn’t a quick process. It was a lot of jumping back and forth between ThingSpeak documentation and code, but I was SUPER excited when it finally uploaded.

                      Aaaah, ok, that makes sense. Thank you!

                  Viewing 7 reply threads
                  • You must be logged in to reply to this topic.