Welcome to EnviroDIY, a community for do-it-yourself environmental science and monitoring. EnviroDIY is part of WikiWatershed, an initiative of Stroud Water Research Center designed to help people advance knowledge and stewardship of fresh water.
New to EnviroDIY? Start here

Using Xbee Cellular Modem with ThingSpeak

Home Forums Mayfly Data Logger Using Xbee Cellular Modem with ThingSpeak

Viewing 50 reply threads
  • Author
    Posts
    • #15724
      Zeke Holloman
      Participant

        Hi all! I’m working on developing a water monitoring station using a Mayfly board, an Xbee cellular modem (XBC LTE-M/NB-IoT Global), the LTEBee adapter, a solar panel, and a Hydros-21 sensor. I saw that you all had already written some Arduino code for logging to ThingSpeak, so I’m using that. I had some issues getting the modem to connect to the internet and getting the RTC to sync to NIST, but I recently figured that out. Now I’m having issues with MQTT. I saw that @srgdamiano and @shicks had responded to some forum topics that contained LTE code, so I figured I’d put my questions here.

        I’ve already setup ThingSpeak and created my topics. When I upload the code, I get a MQTT -4 error (MQTT_CONNECTION_TIMEOUT). I realize this means I haven’t gotten a CONNACK from the broker, but I’m unsure how to remedy the situation. I have attached my code, the pertinent part of my serial monitor output, and the configuration for my cellular Xbee (I’m running the Xbee in Transparent mode). Any help is greatly appreciated! I’m thinking my issue might be that I need to change something in the XCTU settings or potentially that I need to change something in the C++ code inside of the ModularSensors library. Thanks!

         

      • #15729
        Shannon Hicks
        Moderator

          You shouldn’t have to use XCTU for anything, the modem setup part of the sketch programs the Digi module with whatever settings are necessary, based on what you’re trying to do in the sketch.  You’re able to connect to the internet, otherwise the clock syncing function wouldn’t say that your RTC is on time, so I’m guessing your issue is with Thingspeak.  Did you already register a channel with Thingspeak?  If so, you need to put the details of that connection into the appropriate spots on line 201-206 of the sketch you posted above.

        • #15730
          Zeke Holloman
          Participant

            Hi! Thank you for the quick response. Yeah, I already registered a channel with ThingSpeak, and I’ve put the stuff for that into lines 201-206 (the stuff in the sketch above is just placeholders), and running it with those keys and ID’s still doesn’t work. I’d wondered if mqtt.thingspeak.com had been changed to a different URL recently. Is there anything in ThingSpeakPublisher.cpp or ThingSpeakPublisher.h that I need to change?

          • #15731
            Shannon Hicks
            Moderator

              I haven’t used Thingspeak in a very long time, since we have our own data host at MonitorMyWatershed, so I don’t know the specifics about the connections, but I know a few other EnviroDIY members have used it, so maybe one of them can help out.

            • #15732
              Zeke Holloman
              Participant

                Ok! Sounds good; thanks. @neilh20, do you have any experience with ThingSpeak or do you have any advice for this issue?

              • #15733
                neilh20
                Participant

                  Hi @zeke thanks for the qu.  I’ve got the thingspeak working as a POST, but don’t regularly test it. I believe the MQTT is a different route, and I personally haven’t used it.

                  For the Digi Corp LTE Cat-M1 cell modems, I do ensure that they are upgraded to the latest version with XCTU & XCTU board. It is a bit painful, but worked for me. If there is any problem with the Digi LTE modem I force a reset to defaults.

                  The provisioning, customizations all go in the code ~ hologram as you have it, and then in the class should have there is  mqtt.thingspeak.com . ThingSpeakPublisher.cpp

                  I’ll check my notes to see what I did.
                  If nothing else, I would enable the modem stream to see what its doing. Look for

                  // Use this to create a modem if you want to monitor modem communication through
                  // a secondary Arduino stream.  Make sure you install the StreamDebugger
                  // library! https://github.com/vshymanskyy/StreamDebugger
                  #if defined STREAMDEBUGGER_DBG

                  I’ll check out my notes to see what exactly I did on testing.

                • #15735
                  Rick Vogel
                  Participant

                    You might check your ThingSpeak credentials using a desktop MQTT Client just to ensure everything works outside of your project itself as a first step.

                    https://www.mathworks.com/help/thingspeak/use-desktop-mqtt-client-to-subscribe-to-channel-updates.html

                    This example uses MQTT.fx, but I normally use http://mqtt-explorer.com/

                    We are using the Digi LTE-M/NB IoT platform to automate and recover historical COOP sites that have closed. We send 5 minute temp and 15 minute precip. I don’t have any experience using thingspeak, but we have great success with this platform using the micropython programming option built into the Xbees for MQTT. We use a different arduino based platform for the data acquisition and then have the XBee do the heavy lifting for encryption and TLS. It also has capability for FTPS to push remote updates without the need for the Digi Remote Manager. Very capable device.

                    Dashboard –https://www.weather.gov/crh/ccoop

                  • #15736
                    Rick Vogel
                    Participant

                      In the  XBee DE field, type 75B. MQTT port is 1883 (0x75B).

                    • #15737
                      neilh20
                      Participant

                        Hi @zeke – good check by Rick.

                        Looking at your error, it does say connection error so implies it didn’t even connect to thingspeak.

                        I checked this 5months ago and it worked for me. What is your channel number?. This allows a view of if any data has arrived. The channel I used for testing, a bit simplistic  https://thingspeak.com/channels/5940

                        You could also enable some debug and share it.

                        For platformIO this is
                        <div>
                        <div>build_flags =</div>
                        </div>
                        <div>
                        <div>    -DMS_DATAPUBLISHERBASE_DEBUG</div>
                        <div>    -DMS_THINGSPEAKPUBLISHER_DEBUG</div>
                        </div>
                         

                      • #15738
                        Zeke Holloman
                        Participant

                          Hi there! Thank you @neilh20 and @vogelrnws for all of your help! Yeah, based on what I’m seeing, I don’t think its been connecting at all. I have been checking the channel I’m using whenever I’m trying to connect and no data has been popping up in the fields. It’s only got like, 5 fields, so it’s not too complicated either. Sorry, I’m somewhat new to PlatformIO, where do I type this stuff?

                           

                          <div>
                          <div>build_flags =</div>
                          </div>
                          <div>
                          <div>    -DMS_DATAPUBLISHERBASE_DEBUG</div>
                          <div>    -DMS_THINGSPEAKPUBLISHER_DEBUG</div>
                          </div>

                        • #15739
                          Zeke Holloman
                          Participant

                            Also, I was having trouble finding the stream debugger line (#if defined STREAMDEBUGGER_DBG) you were mentioning earlier in the ModularSensors library. Do you know where that would be located?

                          • #15740
                            neilh20
                            Participant

                              I’ll have to get about streamdebugger as heading out the office, back in 2hrs

                              For debug you should see the following in platformIO.ini – add it to the end. It will force a complete recompile

                              <!–more–>
                              [env:mayfly]
                              monitor_speed = 115200
                              board = mayfly
                              platform = atmelavr
                              framework = arduino
                              lib_ldf_mode = deep+
                              lib_ignore =
                              RTCZero

                              build_flags =
                              -DSDI12_EXTERNAL_PCINT
                              -DNEOSWSERIAL_EXTERNAL_PCINT
                              -DMQTT_MAX_PACKET_SIZE=240
                              -DTINY_GSM_RX_BUFFER=64
                              -DTINY_GSM_YIELD_MS=2
                              -DMS_DATAPUBLISHERBASE_DEBUG
                              -DMS_THINGSPEAKPUBLISHER_DEBUG

                            • #15741
                              Zeke Holloman
                              Participant

                                Hey, thanks @neilh20. I added that and uploaded the program and this was the serial monitor output. What does dataPublisherBase refer to?

                                Now running logging_to_thingspeak.ino on Logger logger

                                Using ModularSensors Library version 0.28.5
                                TinyGSM Library version 0.10.9

                                Logger timezone is set to UTC -5
                                RTC timezone is set to UTC
                                CTDcond has a non-unique UUID!
                                CTDtemp has a non-unique UUID!
                                CTDdepth has a non-unique UUID!
                                Battery has a non-unique UUID!
                                12345678-abcd-1234-ef00-1234567890ab -> CTDcond
                                12345678-abcd-1234-ef00-1234567890ab -> CTDtemp
                                12345678-abcd-1234-ef00-1234567890ab -> CTDdepth
                                12345678-abcd-1234-ef00-1234567890ab -> Battery
                                12345678-abcd-1234-ef00-1234567890ab -> decibelMiliWatt

                                Current RTC time is: 2021-07-28T16:08:52-05:00
                                CTDcond has a non-unique UUID!
                                CTDtemp has a non-unique UUID!
                                CTDdepth has a non-unique UUID!
                                Battery has a non-unique UUID!
                                12345678-abcd-1234-ef00-1234567890ab -> CTDcond
                                12345678-abcd-1234-ef00-1234567890ab -> CTDtemp
                                12345678-abcd-1234-ef00-1234567890ab -> CTDdepth
                                12345678-abcd-1234-ef00-1234567890ab -> Battery
                                12345678-abcd-1234-ef00-1234567890ab -> decibelMiliWatt

                                This logger has a variable array with 5 variables, of which 4 come from 2 sensors and 1 are calculated.
                                Logger portion of setup finished.

                                Setting up sensors…
                                Attempting to connect to the internet and synchronize RTC with NIST
                                This may take up to two minutes!
                                Clock already within 5 seconds of time.
                                Putting modem to sleep
                                Setting up file on SD card
                                Data will be saved as logger_2021-07-28.csv
                                Putting processor to sleep
                                ——————————————

                                \/—- Line Saved to SD Card —-\/
                                2021-07-28 16:12:00,0.0,22.30,-4.0,4.882,-69

                                 

                                Sending data to [ 0 ] mqtt.thingspeak.com
                                5 fields will be sent to ThingSpeak <–
                                Topic [ 41 ]: channels/thingSpeakChannelID/publish/thingSpeakChannelKey <–

                                Dumping the TX Buffer <–dataPublisherBase
                                Message [ 96 ]: created_at=2021-07-28T16:12:00-05:00&field1=0.0&field2=22.30&field3=-4.0&field4=4.882&field5=-69 <–
                                Opening MQTT Connection <–
                                MQTT connection failed with state: -4: MQTT_CONNECTION_TIMEOUT
                                Disconnecting from MQTT <–
                                Disconnected after 626 ms <–
                                ——————————————

                                ——————————————

                                \/—- Line Saved to SD Card —-\/
                                2021-07-28 16:13:00,0.0,22.30,-3.8,4.867,0

                                 

                                And from here it just cycles with the same stuff…

                              • #15742
                                neilh20
                                Participant

                                  Hi @zeke-holloman – looks like the ThingSpeak keys and channel need to be added.

                                  Add them to (the following are examples)
                                  const char* thingSpeakMQTTKey =
                                  “Z0G6QX4IXEHKG0PK”; // Your MQTT API Key from Account > MyProfile.
                                  const char* thingSpeakChannelID =
                                  “5940”; // The numeric channel id for your channel eg
                                  const char* thingSpeakChannelKey =
                                  “DHCKWX0HZQLXGIAX”; // The Write API Key for your channel

                                  Your response is
                                  Topic [ 41 ]: channels/thingSpeakChannelID/publish/thingSpeakChannelKey

                                  and it should look something like

                                  Topic [ 38 ]: channels/5940/publish/DHCKWX0HZQLXGIAX <–ThingSpeakPublisher

                                  Followed by this
                                  Dumping the TX Buffer <–dataPublisherBase
                                  Message [ 129 ]: created_at=2021-07-28T19:04:00-08:00&field1=1&field2=2000.000&field3=-0.223&field4=4.1748&field5=4.215&field6=22.20&field7=0.0088 <–ThingSpeakPublisher
                                  Opening MQTT Connection <–ThingSpeakPublisher
                                  MQTT connected after 874 ms <–ThingSpeakPublisher
                                  ThingSpeak topic published! Current state: 0: MQTT_CONNECTED
                                  Disconnecting from MQTT <–ThingSpeakPublisher
                                  Disconnected after 418 ms <–ThingSpeakPublisher

                                • #15744
                                  Zeke Holloman
                                  Participant

                                    Hey @neilh20. Thanks for looking at. Yeah, I already had the keys and the channel ID in the code, I just took them out for the forum. Sorry, I understand that could have been confusing. Other than that, is there anything else you could see it being?

                                  • #15751
                                    Rick Vogel
                                    Participant

                                      @zeke-holloman

                                      I’m not sure if this pertains to your project or not, but when I was looking through the ThingSpeak documentation and going through my test account it had a place to add a MQTT device that would be posting. It issued the device credentials: username, client ID and password…?? Sorry if I’m just causing more confusion, but trying to see what is keeping your unit from the initial broker connection

                                      https://www.mathworks.com/help/thingspeak/mqtt-basics.html#mw_0bf68abc-6c4e-4e2e-8312-e8223a203b71

                                    • #15753
                                      Zeke Holloman
                                      Participant

                                        @vogelrnws

                                         

                                        Hi there! Yeah I registered the MQTT device via ThingSpeak. I used the MQTT Explorer you suggested and I was able to post some data to Thingspeak by typing literal values into the Explorer. This leads me to think that it’s an issue with the Mayfly code.


                                        @neilh20
                                        I’m curious as to how you set up the ThingSpeak using POST. I’m not really committed to the MQTT platform, and I’m curious how you set your’s up.

                                      • #15754
                                        neilh20
                                        Participant

                                          @zeke-holloman  a correction on my part, when I looked through the thingspeak API, both the historical and the MQTT method use the POST to deliver it to thingspeak, however they have a slightly different API.

                                          Initially when looking at your posting I assumed you had removed the keys as it was public, however looking at your your trace it is saying that it doesn’t have the keys in it.

                                          Your response is
                                          Topic [ 41 ]: channels/thingSpeakChannelID/publish/thingSpeakChannelKey

                                          and it should look something like

                                          Topic [ 38 ]: channels/5940/publish/DHCKWX0HZQLXGIAX <–ThingSpeakPublisher

                                          Please confirm that your run has the API keys in.

                                          If that doesn’t work, can you post your platformio.ini

                                          I’m committed to something for the next couple of hours but I can add describe the STREAMDEBUG setup after looking at your platformio.ini

                                           

                                           

                                        • #15755
                                          Zeke Holloman
                                          Participant

                                            Hi @neilh20, thanks for all your help! Yeah, I think I replaced the channel key in the trace with the thingSpeakChannelKey for the forum. I’m looking at the Topic [41] thing and I see the key in the topic channels/…

                                             

                                            Here’s the platformIO file.

                                            [platformio]
                                            src_dir = src/logging_to_thingspeak
                                            ;src_dir = src/LTExBee_FirstConnection

                                            description = ModularSensors example intended for DRWI users with CTD, turbidity, and 2G signal

                                            [env:mayfly]
                                            monitor_speed = 115200
                                            board = mayfly
                                            platform = atmelavr
                                            framework = arduino
                                            lib_ldf_mode = deep+
                                            lib_ignore =
                                            RTCZero
                                            Adafruit NeoPixel
                                            Adafruit GFX Library
                                            Adafruit SSD1306
                                            Adafruit ADXL343
                                            Adafruit STMPE610
                                            Adafruit TouchScreen
                                            Adafruit ILI9341
                                            build_flags =
                                            -DSDI12_EXTERNAL_PCINT
                                            -DNEOSWSERIAL_EXTERNAL_PCINT
                                            -DMQTT_MAX_PACKET_SIZE=240
                                            -DTINY_GSM_RX_BUFFER=64
                                            -DTINY_GSM_YIELD_MS=2
                                            -DMS_DATAPUBLISHERBASE_DEBUG
                                            -DMS_THINGSPEAKPUBLISHER_DEBUG
                                            lib_deps =
                                            envirodiy/EnviroDIY_ModularSensors

                                             

                                          • #15756
                                            neilh20
                                            Participant

                                              @zeke-holloman   interesting, seems like it should work.

                                              When I have strange issues, I delete the .pio/…   and force it to  do a refresh with all the remote libs.  I see you are using the DecagonCTD which recently changed to hydros21, but seems like it should work.

                                              I’ve taken your platformio.ini and program  and adding my thingspeak keys, and will try running. I have a test system  with Digi LTE I’m in the process of switching so I can try your setup

                                              In the meantime

                                              platformio.ini to

                                              build_flags =    -DSTREAMDEBUGGER_DBG   #usually add at end of list

                                              lib_deps =
                                              https://github.com/vshymanskyy/StreamDebugger  ;Debug when needed

                                              logging_to_thinkgspeak :  look for DigiXBeeCellularTransparent  and make changes

                                              // Create the modem object
                                              <span style=”color: #ff0000;”>#if defined STREAMDEBUGGER_DBG</span>
                                              <span style=”color: #ff0000;”>#include <StreamDebugger.h></span>
                                              <span style=”color: #ff0000;”>StreamDebugger modemDebugger(modemSerial, STANDARD_SERIAL_OUTPUT);</span>
                                              <span style=”color: #ff0000;”>#define modemSerHw modemDebugger</span>
                                              <span style=”color: #ff0000;”>#else</span>
                                              <span style=”color: #ff0000;”>#define modemSerHw modemSerial</span>
                                              <span style=”color: #ff0000;”>#endif  // STREAMDEBUGGER_DBG</span>
                                              DigiXBeeCellularTransparent modemXBCT(&<span style=”color: #ff0000;”>modemSerHw</span>, modemVccPin, modemStatusPin,
                                              useCTSforStatus, modemResetPin,
                                              modemSleepRqPin, apn);

                                            • #15757
                                              neilh20
                                              Participant

                                                @zeke-holloman – I have no idea why my previous post came out so badly, and it won’t allow me to edit it!!. My usual way of recovering strange posts.

                                                It should be interesting to see what you get with your cell modem.

                                                Using your sketch, which references the latest ModularSensors

                                                Using ModularSensors Library version 0.30.0
                                                TinyGSM Library version 0.11.4

                                                My cell modem which is running on Verizon wouldn’t connect.  The modem issues are  technical and I believe understood, and have been discussed.

                                                This specific modem has been working no problem in my fork  which has modifications for managing the startup of the cell modem and works for the cell company I’ve been using …. my fork currently is at ModularSensors 0.28.5

                                                I did try compiling your program in my fork, but there is some changes in 0.30.0 that aren’t backward compatible, so I need to upgrade before I can try this.

                                                Still with your software, referncing MS 0.30.0  I switched the modem to Digi WiFi  S6B, which I use a lot in testing,  and it worked no problem.

                                                Once connected to an TCP/IP link – whether over cell or wifi – the passage of data is the same, though Modem AT cmds may be a little different.

                                                So this is what I get

                                              • #15759
                                                Zeke Holloman
                                                Participant

                                                  Hi there @neilh20. Thanks for your response, sorry I couldn’t get to it until now. I spent some time on the code, and I tried the code with the 0.28.5 version of the EnviroDIY ModularSensors library just to see if anything changed. It still isn’t uploading to ThingSpeak though. I tried the 0.30.0 version of the ModularSensors library, alongside the StreamDebugger as well, here’s the output of the code (excluding the ThingSpeak data). It doesn’t look that much different.

                                                   

                                                  Windows PowerShell
                                                  Copyright (C) Microsoft Corporation. All rights reserved.

                                                  Try the new cross-platform PowerShell https://aka.ms/pscore6

                                                  PS C:\Users\waked\OneDrive – East Carolina University\Documents\PlatformIO\Projects\LearnEnviroDIYcode-master> pio device monitor –port COM10 –baud 115200
                                                  — Available filters and text transformations: colorize, debug, default, direct, hexlify, log2file, nocontrol, printable, send_on_enter, time
                                                  — More details at http://bit.ly/pio-monitor-filters
                                                  — Miniterm on COM10 115200,8,N,1 —
                                                  — Quit: Ctrl+C | Menu: Ctrl+T | Help: Ctrl+T followed by Ctrl+H —
                                                  Now running logging_to_thingspeak5.ino on Logger logger

                                                  Using ModularSensors Library version TinyGSM Library version 0.11.4
                                                  Current RTC time is: 2021-08-02T12:16:53-05:00
                                                  CTDcond has a non-unique UUID!
                                                  CTDtemp has a non-unique UUID!
                                                  CTDdepth has a non-unique UUID!
                                                  Battery has a non-unique UUID!
                                                  12345678-abcd-1234-ef00-1234567890ab -> CTDcond
                                                  12345678-abcd-1234-ef00-1234567890ab -> CTDtemp
                                                  12345678-abcd-1234-ef00-1234567890ab -> CTDdepth
                                                  12345678-abcd-1234-ef00-1234567890ab -> Battery
                                                  12345678-abcd-1234-ef00-1234567890ab -> decibelMiliWatt

                                                  This logger has a variable array with 5 variables, of which 4 come from 2 sensors and 1 are calculated.

                                                   

                                                  Setting up sensors…
                                                  Attempting to connect to the internet and synchronize RTC with NIST
                                                  This may take up to two minutes!
                                                  OKCN163.97.697.6
                                                  OKCN163.97.6
                                                  OKCN163.97.6
                                                  OKCN163.97.6
                                                  OKCN163.97.6

                                                  Clock already within 5 seconds of time.
                                                  Putting modem to sleep
                                                  Setting up file on SD card
                                                  Data will be saved as logger_2021-08-02.csv
                                                  Putting processor to sleep
                                                  ——————————————

                                                  \/—- Line Saved to SD Card —-\/
                                                  2021-08-02 12:20:00,0.0,22.40,-3.2,4.897,-81

                                                   

                                                  OKCN3.23.121
                                                  Sending data to [ 0 ] mqtt.thingspeak.com
                                                  MQTT connection failed with state: -4: MQTT_CONNECTION_TIMEOUT

                                                  ——————————————

                                                  ——————————————

                                                  \/—- Line Saved to SD Card —-\/
                                                  2021-08-02 12:21:00,0.0,22.40,-2.7,4.897,0

                                                  OKCN3.23.121
                                                  Sending data to [ 0 ] mqtt.thingspeak.com
                                                  MQTT connection failed with state: -4: MQTT_CONNECTION_TIMEOUT

                                                   

                                                  Additionally, here’s the updated version of the code with the StreamDebugger stuff.

                                                  That’s very strange that the code worked with the Digi Wifi module. Any other suggestions are greatly appreciated! And if you see anything that I did wrong with the code, please let me know! I’m still rather new to this.

                                                • #15761
                                                  Zeke Holloman
                                                  Participant

                                                    Hey @neilh20. I just had a thought. The fact that the software works with the Digi WiFi S6B. Do you think that means its an issue with the actual cellular connection as opposed to the code?

                                                  • #15762
                                                    neilh20
                                                    Participant

                                                      hello @zeke-holloman. Just looking through it, and yes thats my assumption at the moment is that its the cellular side that is not connecting.
                                                      There are 2 parts 1) getting internet & time 2) thingspeak.
                                                      1) If the cell phone connects for NIST time, then it can connect with the internet, and the cell modem account setting is good.

                                                      2) if thingspeak then doesn’t connect its something to do with the account ino.

                                                      I’m just looking at your posted new file

                                                      The STREAMDEBUGGER_DBG is a bit strange, it should really be this

                                                      The log you posted with some basic cell phone interface is looking very minimal. Possibly also compile it with

                                                      -DMS_DATAPUBLISHERBASE_DEBUG
                                                      -DMS_THINGSPEAKPUBLISHER_DEBUG

                                                      -DSTREAMDEBUGGER_DBG

                                                      .
                                                      I was thinking of creating a branch off envirodiy (master) – and seeing if it can be referenced for an update to the cell phone interface so it can show more detail. Gotta try it out.

                                                    • #15763
                                                      Rick Vogel
                                                      Participant

                                                        @neilh20 and @zeke-holloman

                                                        Gonna let you all work through this on the mayfly code using the xbee transparent mode. If at some point you would like to use the micro-python option in the xbee I can provide some code to just use simple serial.println() on the mayfly to send payload to the xbee and it will handle all the MQTT stuff in the background.

                                                      • #15764
                                                        Zeke Holloman
                                                        Participant

                                                          Hi there @vogelrnws. Honestly, It’d be awesome if you could send that code. I’m still considering my options on the method to publish the data, but that could be helpful!

                                                           


                                                          @neilh20
                                                          , ok! I’ll try that. For some reason (not really sure why), the code all of a sudden started saying that it couldn’t connect to the internet for clock sync. I’m looking into that, so I’ll get back to you once I fix that. Thank you again!

                                                        • #15765
                                                          neilh20
                                                          Participant

                                                            Hello @rick-vogel – sounds fascinating & fun to use python in the Xbee.  Though for ModularSensors there are layers of software that allow the physical modem to be abstracted so it could be possible to use another LTE when it becomes available (0.30.0 seems to have hooks for a new LTE modem SIM7080G) – through TinyGSM. As a Class package that @srgdamiano has put together its quite comprehensive – though as with all abstractions it has upsides and downsides.   I think this is working through what it takes to figure out the control flow, which would be needed whatever route is taken. In my fork (with some LTE interface changes) it has worked.


                                                            @zeke-holloman
                                                            , I’d got pulled off onto something else, now I’ve added the STREAMDEBUGGER_DBG to your “Logging_to_Thingskeak5.ino”and removed my extra code that I had and about to test it – but I have to get another dataplan going to test it – I use https://dataplans.digikey.com/ that has a 50Mbytes/month plans,

                                                          • #15766
                                                            neilh20
                                                            Participant

                                                              @zeke-holloman,  I figured out that the other day I hadn’t changed the apn to the source I was using.

                                                              So with your code, my apn and thingspeak API, using tera-term 4.105 with TimeStamp and the following changes I got a good response from  thingspeak

                                                              #include <ModularSensors.h> //pre 0.30.0 was <LoggerBase.h></div>

                                                              The core thingspeak looks like this

                                                              I have a lot more debug, and better scale-ability built into my fork https://github.com/neilh10/ModularSensors but the above worked for me.

                                                            • #15767
                                                              neilh20
                                                              Participant

                                                                https://thingspeak.com/channels/5940 shows a data point at Aug 02 2021 15:17 GMT-0700  Yeah!!

                                                              • #15768
                                                                Rick Vogel
                                                                Participant

                                                                  Here are the python files if you ever decide to look at that option. Please feel free to manipulate however you need to.

                                                                  • Open up the main.py file in a ‘python’ editor and use the credentials you get from the MQTT device you created on thingspeak, not your API keys. Save file.
                                                                  • Within Digi XTCU you will need to go to File System Manager on the Xbee and go into the “lib” folder.
                                                                  • Create a folder named umqtt. (right click, create directory, name it, enter) Place the simple.py file in that folder.
                                                                  • Go back to the main /flash directory (should show cert and lib folders) and transfer the main.py file into that location.

                                                                  This main file does a few different things. Sorry, I didn’t want to completely rewrite the file so I used a lot of what I already had in place and deleted the FTPS portions.

                                                                  • It will recognize data that it needs to publish from the mayfly be adding 2 things.
                                                                    • #P on the front of the data string (no space between this and the actual data)
                                                                    • ‘\n’ on the end of the data string (println() from mayfly for each data string you want to send works)
                                                                      • so….. #Pfield1=xx&field2=xx&field3=xx……\r\n
                                                                      • python uses the \n to separate the strings and then trims the \r later.
                                                                      • you can send multiple strings at once as long as each one has these two identifiers on the front and back and this file will split them into individual posts.
                                                                  • Because there may be times where you loose the cell tower, this file creates a couple of files to log the data in case it can’t post. There is a topic file and a message file. Each time a post comes in for publishing it will first back it up to the files. Once it posts then it will clear the file. Does the same thing for power outages. If you have backed up messages that still need to post and you lose power, it will read those logs on boot and place them in a buffer to send. Once they all send it will clear the files.

                                                                  within the main.py there are some print statements I had in there while troubleshooting this connection. You can comment them out if needed. I also connected my XCTU profile if you want to dump this to the XBee for settings establishment (profile -> apply profile). I’m on 11417 firmware so you may want to update to that if you want to use it. The only thing you should need to change would be your apn. I use twilio super sims and the apn is different.

                                                                  Please let me know if you have any questions. I know this is a fair bit of info. more good info on the python aspect of XBee is here:

                                                                  https://www.digi.com/resources/documentation/Digidocs/90002219/#reference/r_access_data.htm%3FTocPath%3DAccess%2520file%2520system%2520in%2520MicroPython%7C_____2

                                                                • #15769
                                                                  Zeke Holloman
                                                                  Participant

                                                                    @vogelrnws Thank you for sending that. I’ll be sure to take a look at it if ThingSpeak doesn’t work.

                                                                     


                                                                    @neilh20
                                                                    I tried running the code with your ModularSensors library, and it still didn’t work. The apn I’m using is hologram. If I am using a Hologram SIM card, is that correct, or should I change the source to something else? Also, what is Tera-term with Timestamp, and is that something I should consider using?

                                                                    And also, one more question. Is ThingSpeak with API different that ThingSpeak with MQTT? Right now, it looks like I’m using the MQTT connection.

                                                                  • #15770
                                                                    Rick Vogel
                                                                    Participant

                                                                      @zeke-holloman

                                                                      Everything I was talking about is MQTT with Thingspeak.  I tested it on my Thingspeak connection I created. It was just allowing the micro-python code in the Xbee to handle all the network and MQTT items vice using the AT commands from the MayFly to the Xbee as it looks like is happening in the enviroDIY libraries.

                                                                    • #15771
                                                                      neilh20
                                                                      Participant

                                                                        @zeke-holloman do you want to send me the thingspeak details on my email neilh20@wLLw.net and I’ll try them in my setting.  I have a historical thingspeak account which has exceeded its free quota, so I can’t create more devices.

                                                                        The hologram account should work with the Xbee LTE ( I have an XB3-CA2-UT-001) if setup, and I started with hologram.  If it works for getting time, should work for any other connection.  If you post the trace can see if its getting good response .

                                                                        @vogelrnws  gosh sounds interesting, especially as you are logging the readings. I couldn’t see the python files – could you provide a link.

                                                                         

                                                                      • #15772
                                                                        Rick Vogel
                                                                        Participant

                                                                          @neilh20

                                                                          I’ll put the files up on a google drive and send a link for a folder to download here in a bit.

                                                                          Yes, micro-python gives you lots of option. We use TLS1.2 encryption of our data, run a broker at one of our regional headquarters and also have a FTP-Secure server to remotely update both the XBee and the Teensy 3.5 processor I use as the datalogger. The Teensy runs a special bootloader that allows it to boot off the SD Card file. So I can pull the new file from the server with the XBee and then transfer via the serial port to the Teensy as it writes it to the card. Reboot and boom, new program is running.

                                                                        • #15776
                                                                          neilh20
                                                                          Participant

                                                                            @vogelrnws  sounds great .. the Teensy 3.5 is a great board and amazing support. If you’ve figured out remote upgrade as well, very noce.  I wonder if you would create a separate thread for it ~ perhaps “using the Teensy 3.5”. One of the biggest issues for me is always how to make a system, low power and solar powered.

                                                                          • #15780
                                                                            Zeke Holloman
                                                                            Participant

                                                                              @neilh20 Ok, that sounds good. I just sent the details to your email. Let me know if it still works.

                                                                            • #15781
                                                                              Rick Vogel
                                                                              Participant

                                                                                @neilh

                                                                                Teensy 3.5 topic set up.

                                                                                @Both

                                                                                google drive link for the XBee files if either one of you want them in the future.

                                                                                https://drive.google.com/drive/folders/1cr6PCJJRH8-qYIqOKSRQzYnqL8qg4nC5?usp=sharing

                                                                                 

                                                                              • #15801
                                                                                Zeke Holloman
                                                                                Participant

                                                                                  @vogelrnws I was spending some time looking at the cellular stuff and I decided I’d try to use the python example you used, but I can’t seem to access the files through the Google Drive. Also, I don’t think I currently have a python editor. What is a good one that you would suggest? Thanks

                                                                                • #15802
                                                                                  Rick Vogel
                                                                                  Participant

                                                                                    it seems I have to add your contact email to allow the share. I’ll see if I can post the zip file. It didn’t like the .py files before

                                                                                  • #15803
                                                                                    Zeke Holloman
                                                                                    Participant

                                                                                      @vogelrnws Ok, thank you. If you need my email, zholloman54@gmail.com

                                                                                    • #15804
                                                                                      Rick Vogel
                                                                                      Participant
                                                                                      • #15805
                                                                                        Zeke Holloman
                                                                                        Participant

                                                                                          Awesome, thank you. That worked

                                                                                        • #15806
                                                                                          Zeke Holloman
                                                                                          Participant

                                                                                            @vogelrnws Hi there! So I uploaded the files onto the Xbee and used a python editor to change the main.py file. Is there any arduino code you would recommend using alongside this approach?

                                                                                          • #15807
                                                                                            Rick Vogel
                                                                                            Participant

                                                                                              Any arduino code you would like to use. The key is to identify which serial port you have connected to the XBee. For instance if you have it connected to Serial1 then you would want to print out the statements you want sent to Thingspeak like so…

                                                                                              Topic posting to includes multiple fields

                                                                                              Serial1.println(#Pfield1=60&field2=45)

                                                                                              Topic posting to is field specific to field1:

                                                                                              Serial1.println(#P60)

                                                                                              The #P is the identifier the xbee python code is looking for to identify the data you want sent. the println() includes the \r\n characters that allow the python code to separate multiple post request coming in one after the other.

                                                                                            • #15808
                                                                                              Zeke Holloman
                                                                                              Participant

                                                                                                @vogelrnws Ah, ok! Awesome, thank you. And is there a place to put the apn in the main code?

                                                                                              • #15809
                                                                                                Rick Vogel
                                                                                                Participant

                                                                                                  No, the apn would need to be loaded via XCTU or you can place it in the python code before the cellular connection action…

                                                                                                   

                                                                                                  xbee.atcmd(“AN”, “hologram”) —–or whatever your apn is

                                                                                                  conn = network.Cellular()

                                                                                                • #15816
                                                                                                  neilh20
                                                                                                  Participant

                                                                                                    @zeke-holloman  I’ve been out the office. Just to summarize our offline investigation – I got a good POST using your thingspeak settings. It seem to me that its your modem data plan that isn’t connecting in some way. Maybe there is a better error indication from DigiXBeeCellularTransparent, but its an evolving area to make it easy.

                                                                                                  • #15818
                                                                                                    Zeke Holloman
                                                                                                    Participant

                                                                                                      Ok, thanks Neil. I’ll look into it.

                                                                                                    • #15899
                                                                                                      Zeke Holloman
                                                                                                      Participant

                                                                                                        @neilh20. Hey Neil! Sorry it’s been a while, but just a quick update. I was able to get it to work. I ended up purchasing a new XBee cellular modem (same type) alongside the Digi IoT Development Kit. Works great now!

                                                                                                         

                                                                                                        I’m thinking the issue was either that I had set up the modem with a board that wasn’t the dedicated XCTU board (I was using the Xbee Grove Development board). Either that, or I just messed up the set-up with the Xbee from the get-go.

                                                                                                         

                                                                                                        Thanks for all your help!

                                                                                                      • #15900
                                                                                                        neilh20
                                                                                                        Participant

                                                                                                          @zeke-holloman  great glad to hear it.

                                                                                                          Which modem did you buy and where did you get it?. I thought they were in short supply,  and I’ve been keeping my eye open for them.

                                                                                                          I’ve been using the Digi LTE XB3-C-A2-UT-001, which needs the the latest XCTU modem board to upgrade as its is a complex process.

                                                                                                        • #15901
                                                                                                          Rick Vogel
                                                                                                          Participant

                                                                                                            That is the version of XBee I use as well. They are very short supply. I only have some cause we bought a large supply of them bout a year ago.  The dev board I use with mine to upgrade the cellular firmware and normal XCTU operations is the XBIB-CU-TH. Digi-Key has 200+ of those.

                                                                                                        Viewing 50 reply threads
                                                                                                        • You must be logged in to reply to this topic.