Welcome to EnviroDIY, a community for do-it-yourself environmental science and monitoring. EnviroDIY is part of Home › Forums › Mayfly Data Logger › Using Xbee Cellular Modem with ThingSpeak
Tagged: cellular, LTE, PlatformIO, Xbee, XCTU
- This topic has 50 replies, 4 voices, and was last updated 2021-09-14 at 11:17 AM by
Rick Vogel.
-
AuthorPosts
-
-
2021-07-27 at 2:58 PM #15724Hi all! I’m working on developing a water monitoring station using a Mayfly board, an Xbee cellular modem (XBC LTE-M/NB-IoT Global), the LTEBee adapter, a solar panel, and a Hydros-21 sensor. IHi all! I’m working on developing a water monitoring station using a Mayfly board, an Xbee cellular modem (XBC LTE-M/NB-IoT Global), the LTEBee adapter, a solar panel, and a Hydros-21 sensor. I saw that you all had already written some Arduino code for logging to ThingSpeak, so I’m using that. I had some issues getting the modem to connect to the internet and getting the RTC to sync to NIST, but I recently figured that out. Now I’m having issues with MQTT. I saw that @srgdamiano and @shicks had responded to some forum topics that contained LTE code, so I figured I’d put my questions here.
I’ve already setup ThingSpeak and created my topics. When I upload the code, I get a MQTT -4 error (MQTT_CONNECTION_TIMEOUT). I realize this means I haven’t gotten a CONNACK from the broker, but I’m unsure how to remedy the situation. I have attached my code, the pertinent part of my serial monitor output, and the configuration for my cellular Xbee (I’m running the Xbee in Transparent mode). Any help is greatly appreciated! I’m thinking my issue might be that I need to change something in the XCTU settings or potentially that I need to change something in the C++ code inside of the ModularSensors library. Thanks!
123456789101112131415161718Setting up sensors...Attempting to connect to the internet and synchronize RTC with NISTThis may take up to two minutes!Clock already within 5 seconds of time.Putting modem to sleepSetting up file on SD cardData will be saved as logger_2021-07-27.csvPutting processor to sleep------------------------------------------\/---- Line Saved to SD Card ----\/2021-07-27 13:40:00,0.0,24.30,-9.8,4.821,-51Sending data to [ 0 ] mqtt.thingspeak.comMQTT connection failed with state: -4: MQTT_CONNECTION_TIMEOUT123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345/** =========================================================================* @file logging_to_ThingSpeak.ino* @brief Example logging data and publishing to ThingSpeak.** @author Sara Geleskie Damiano <sdamiano@stroudcenter.org>* @copyright (c) 2017-2020 Stroud Water Research Center (SWRC)* and the EnviroDIY Development Team* This example is published under the BSD-3 license.** Build Environment: Visual Studios Code with PlatformIO* Hardware Platform: EnviroDIY Mayfly Arduino Datalogger** DISCLAIMER:* THIS CODE IS PROVIDED "AS IS" - NO WARRANTY IS GIVEN.* ======================================================================= */// ==========================================================================// Defines for the Arduino IDE// NOTE: These are ONLY needed to compile with the Arduino IDE.// If you use PlatformIO, you should set these build flags in your// platformio.ini// ==========================================================================/** Start [defines] */#ifndef TINY_GSM_RX_BUFFER#define TINY_GSM_RX_BUFFER 64#endif#ifndef TINY_GSM_YIELD_MS#define TINY_GSM_YIELD_MS 2#endif#ifndef MQTT_MAX_PACKET_SIZE#define MQTT_MAX_PACKET_SIZE 240#endif#define TINY_GSM_MODEM_XBEE/** End [defines] */// ==========================================================================// Include the libraries required for any data logger// ==========================================================================/** Start [includes] */// The Arduino library is needed for every Arduino program.#include <Arduino.h>// EnableInterrupt is used by ModularSensors for external and pin change// interrupts and must be explicitly included in the main program.#include <EnableInterrupt.h>// To get all of the base classes for ModularSensors, include LoggerBase.// NOTE: Individual sensor definitions must be included separately.#include <LoggerBase.h>/** End [includes] */// ==========================================================================// Data Logging Options// ==========================================================================/** Start [logging_options] */// The name of this program fileconst char* sketchName = "logging_to_thingspeak.ino";// Logger ID, also becomes the prefix for the name of the data file on SD cardconst char* LoggerID = "logger";// How frequently (in minutes) to log dataconst uint8_t loggingInterval = 1;// Your logger's timezone.const int8_t timeZone = -5; // Eastern Standard Time// NOTE: Daylight savings time will not be applied! Please use standard time!// Set the input and output pins for the logger// NOTE: Use -1 for pins that do not applyconst int32_t serialBaud = 115200; // Baud rate for debuggingconst int8_t greenLED = 8; // Pin for the green LEDconst int8_t redLED = 9; // Pin for the red LEDconst int8_t buttonPin = 21; // Pin for debugging mode (ie, button pin)const int8_t wakePin = A7; // MCU interrupt/alarm pin to wake from sleep// Set the wake pin to -1 if you do not want the main processor to sleep.// In a SAMD system where you are using the built-in rtc, set wakePin to 1const int8_t sdCardPwrPin = -1; // MCU SD card power pinconst int8_t sdCardSSPin = 12; // SD card chip select/slave select pinconst int8_t sensorPowerPin = 22; // MCU pin controlling main sensor power/** End [logging_options] */// ==========================================================================// Wifi/Cellular Modem Options// ==========================================================================/** Start [xbee_cell_transparent] */// For any Digi Cellular XBee's// NOTE: The u-blox based Digi XBee's (3G global and LTE-M global) can be used// in either bypass or transparent mode, each with pros and cons// The Telit based Digi XBees (LTE Cat1) can only use this mode.#include <modems/DigiXBeeCellularTransparent.h>// NOTE: Extra hardware and software serial ports are created in the "Settings// for Additional Serial Ports" sectionHardwareSerial& modemSerial = Serial1; // Use hardware serial if possibleconst int32_t modemBaud = 9600; // All XBee's use 9600 by default// Modem Pins - Describe the physical pin connection of your modem to your board// NOTE: Use -1 for pins that do not apply// The pin numbers here are for a Digi XBee with a Mayfly and LTE adapter// For options https://github.com/EnviroDIY/LTEbee-Adapter/edit/master/README.mdconst int8_t modemVccPin = -1; // MCU pin controlling modem power// Option: modemVccPin = A5, if Mayfly SJ7 is// connected to the ASSOC pinconst int8_t modemStatusPin = 13; // MCU pin used to read modem status// NOTE: If possible, use the <code>STATUS/SLEEP_not</code> (XBee pin 13) for status, but// the CTS pin can also be used if necessaryconst bool useCTSforStatus = false; // Flag to use the CTS pin for statusconst int8_t modemResetPin = 20; // MCU pin connected to modem reset pinconst int8_t modemSleepRqPin = 23; // MCU pin for modem sleep/wake requestconst int8_t modemLEDPin = redLED; // MCU pin connected an LED to show modem// status// Network connection informationconst char* apn = "hologram"; // APN for GPRS connection// Create the modem objectDigiXBeeCellularTransparent modemXBCT(&modemSerial, modemVccPin, modemStatusPin,useCTSforStatus, modemResetPin,modemSleepRqPin, apn);// Create an extra reference to the modem by a generic nameDigiXBeeCellularTransparent modem = modemXBCT;/** End [xbee_cell_transparent] */// ==========================================================================// ==========================================================================// Using the Processor as a Sensor// ==========================================================================/** Start [processor_sensor] */#include <sensors/ProcessorStats.h>// Create the main processor chip "sensor" - for general metadataconst char* mcuBoardVersion = "v0.5b";ProcessorStats mcuBoard(mcuBoardVersion);/** End [processor_sensor] */// ==========================================================================// Maxim DS3231 RTC (Real Time Clock)// ==========================================================================/** Start [ds3231] */#include <sensors/MaximDS3231.h>// Create a DS3231 sensor objectMaximDS3231 ds3231(1);/** End [ds3231] */// ==========================================================================// Meter Hydros 21 Conductivity, Temperature, and Depth Sensor// ==========================================================================/** Start [decagon_ctd] */#include <sensors/DecagonCTD.h>const char* CTDSDI12address = "1"; // The SDI-12 Address of the CTDconst uint8_t CTDNumberReadings = 6; // The number of readings to averageconst int8_t SDI12Power = sensorPowerPin; // Power pin (-1 if unconnected)const int8_t SDI12Data = 7; // The SDI12 data pin// Create a Decagon CTD sensor objectDecagonCTD ctd(*CTDSDI12address, SDI12Power, SDI12Data, CTDNumberReadings);/** End [decagon_ctd] */// ==========================================================================// Creating the Variable Array[s] and Filling with Variable Objects// ==========================================================================/** Start [variable_arrays] */Variable* variableList[] = {new DecagonCTD_Cond(&ctd, "12345678-abcd-1234-ef00-1234567890ab"),new DecagonCTD_Temp(&ctd, "12345678-abcd-1234-ef00-1234567890ab"),new DecagonCTD_Depth(&ctd, "12345678-abcd-1234-ef00-1234567890ab"),new ProcessorStats_Battery(&mcuBoard,"12345678-abcd-1234-ef00-1234567890ab"),new Modem_RSSI(&modem, "12345678-abcd-1234-ef00-1234567890ab")};// Count up the number of pointers in the arrayint variableCount = sizeof(variableList) / sizeof(variableList[0]);// Create the VariableArray objectVariableArray varArray;/** End [variable_arrays] */// ==========================================================================// The Logger Object[s]// ==========================================================================/** Start [loggers] */// Create a logger instanceLogger dataLogger;/** End [loggers] */// ==========================================================================// Creating Data Publisher[s]// ==========================================================================// Create a channel with fields on ThingSpeak in advance// The fields will be sent in exactly the order they are in the variable array.// Any custom name or identifier given to the field on ThingSpeak is irrelevant.// No more than 8 fields of data can go to any one channel. Any fields beyond// the eighth in the array will be ignored.const char* thingSpeakMQTTKey ="custom_key_here"; // Your MQTT API Key from Account > MyProfile.const char* thingSpeakChannelID ="custom_id_here"; // The numeric channel id for your channelconst char* thingSpeakChannelKey ="custom_write_key_here"; // The Write API Key for your channel// Create a data publisher for ThingSpeak#include <publishers/ThingSpeakPublisher.h>ThingSpeakPublisher TsMqtt;/** End [loggers] */// ==========================================================================// Working Functions// ==========================================================================/** Start [working_functions] */// Flashes the LED's on the primary boardvoid greenredflash(uint8_t numFlash = 4, uint8_t rate = 75) {for (uint8_t i = 0; i < numFlash; i++) {digitalWrite(greenLED, HIGH);digitalWrite(redLED, LOW);delay(rate);digitalWrite(greenLED, LOW);digitalWrite(redLED, HIGH);delay(rate);}digitalWrite(redLED, LOW);}// Reads the battery voltage// NOTE: This will actually return the battery level from the previous update!float getBatteryVoltage() {if (mcuBoard.sensorValues[0] == -9999) mcuBoard.update();return mcuBoard.sensorValues[0];}/** End [working_functions] */// ==========================================================================// Arduino Setup Function// ==========================================================================/** Start [setup] */void setup() {// Start the primary serial connectionSerial.begin(serialBaud);// Print a start-up note to the first serial portSerial.print(F("Now running "));Serial.print(sketchName);Serial.print(F(" on Logger "));Serial.println(LoggerID);Serial.println();Serial.print(F("Using ModularSensors Library version "));Serial.println(MODULAR_SENSORS_VERSION);Serial.print(F("TinyGSM Library version "));Serial.println(TINYGSM_VERSION);Serial.println();// Start the serial connection with the modemmodemSerial.begin(modemBaud);// Set up pins for the LED'spinMode(greenLED, OUTPUT);digitalWrite(greenLED, LOW);pinMode(redLED, OUTPUT);digitalWrite(redLED, LOW);// Blink the LEDs to show the board is on and starting upgreenredflash();// Set the timezones for the logger/data and the RTC// Logging in the given time zoneLogger::setLoggerTimeZone(timeZone);// It is STRONGLY RECOMMENDED that you set the RTC to be in UTC (UTC+0)Logger::setRTCTimeZone(0);// Attach the modem and information pins to the loggerdataLogger.attachModem(modem);modem.setModemLED(modemLEDPin);dataLogger.setLoggerPins(wakePin, sdCardSSPin, sdCardPwrPin, buttonPin,greenLED);// Begin the variable array[s], logger[s], and publisher[s]varArray.begin(variableCount, variableList);dataLogger.begin(LoggerID, loggingInterval, &varArray);TsMqtt.begin(dataLogger, &modem.gsmClient, thingSpeakMQTTKey,thingSpeakChannelID, thingSpeakChannelKey);// Note: Please change these battery voltages to match your battery// Set up the sensors, except at lowest battery levelif (getBatteryVoltage() > 3.4) {Serial.println(F("Setting up sensors..."));varArray.setupSensors();}// Sync the clock if it isn't valid or we have battery to spareif (getBatteryVoltage() > 3.55 || !dataLogger.isRTCSane()) {// Synchronize the RTC with NIST// This will also set up the modemdataLogger.syncRTC();}// Create the log file, adding the default header to it// Do this last so we have the best chance of getting the time correct and// all sensor names correct// Writing to the SD card can be power intensive, so if we're skipping// the sensor setup we'll skip this too.if (getBatteryVoltage() > 3.4) {Serial.println(F("Setting up file on SD card"));dataLogger.turnOnSDcard(true); // true = wait for card to settle after power updataLogger.createLogFile(true); // true = write a new headerdataLogger.turnOffSDcard(true); // true = wait for internal housekeeping after write}// Call the processor sleepSerial.println(F("Putting processor to sleep"));dataLogger.systemSleep();}/** End [setup] */// ==========================================================================// Arduino Loop Function// ==========================================================================/** Start [loop] */// Use this short loop for simple data logging and sendingvoid loop() {// Note: Please change these battery voltages to match your battery// At very low battery, just go back to sleepif (getBatteryVoltage() < 3.4) {dataLogger.systemSleep();}// At moderate voltage, log data but don't send it over the modemelse if (getBatteryVoltage() < 3.55) {dataLogger.logData();}// If the battery is good, send the data to the worldelse {dataLogger.logDataAndPublish();}}/** End [loop] */ -
2021-07-27 at 4:09 PM #15729You shouldn’t have to use XCTU for anything, the modem setup part of the sketch programs the Digi module with whatever settings are necessary, based on what you’re trying to do in the sketYou shouldn’t have to use XCTU for anything, the modem setup part of the sketch programs the Digi module with whatever settings are necessary, based on what you’re trying to do in the sketch. You’re able to connect to the internet, otherwise the clock syncing function wouldn’t say that your RTC is on time, so I’m guessing your issue is with Thingspeak. Did you already register a channel with Thingspeak? If so, you need to put the details of that connection into the appropriate spots on line 201-206 of the sketch you posted above.
-
2021-07-27 at 4:16 PM #15730Hi! Thank you for the quick response. Yeah, I already registered a channel with ThingSpeak, and I’ve put the stuff for that into lines 201-206 (the stuff in the sketch above is just placeholdersHi! Thank you for the quick response. Yeah, I already registered a channel with ThingSpeak, and I’ve put the stuff for that into lines 201-206 (the stuff in the sketch above is just placeholders), and running it with those keys and ID’s still doesn’t work. I’d wondered if mqtt.thingspeak.com had been changed to a different URL recently. Is there anything in ThingSpeakPublisher.cpp or ThingSpeakPublisher.h that I need to change?
-
2021-07-27 at 4:21 PM #15731I haven’t used Thingspeak in a very long time, since we have our own data host at MonitorMyWatershed, so I don’t know the specifics about the connections, but I know a few other EnviroDIYI haven’t used Thingspeak in a very long time, since we have our own data host at MonitorMyWatershed, so I don’t know the specifics about the connections, but I know a few other EnviroDIY members have used it, so maybe one of them can help out.
-
2021-07-27 at 5:10 PM #15732
Ok! Sounds good; thanks. @neilh20, do you have any experience with ThingSpeak or do you have any advice for this issue?
-
2021-07-27 at 10:41 PM #15733Hi @zeke thanks for the qu. I’ve got the thingspeak working as a POST, but don’t regularly test it. I believe the MQTT is a different route, and I personally haven’t used it.
For
Hi @zeke thanks for the qu. I’ve got the thingspeak working as a POST, but don’t regularly test it. I believe the MQTT is a different route, and I personally haven’t used it.For the Digi Corp LTE Cat-M1 cell modems, I do ensure that they are upgraded to the latest version with XCTU & XCTU board. It is a bit painful, but worked for me. If there is any problem with the Digi LTE modem I force a reset to defaults.
The provisioning, customizations all go in the code ~ hologram as you have it, and then in the class should have there is mqtt.thingspeak.com . ThingSpeakPublisher.cpp
I’ll check my notes to see what I did.
If nothing else, I would enable the modem stream to see what its doing. Look for// Use this to create a modem if you want to monitor modem communication through
// a secondary Arduino stream. Make sure you install the StreamDebugger
// library! https://github.com/vshymanskyy/StreamDebugger
#if defined STREAMDEBUGGER_DBGI’ll check out my notes to see what exactly I did on testing.
-
2021-07-28 at 3:49 PM #15735You might check your ThingSpeak credentials using a desktop MQTT Client just to ensure everything works outside of your project itself as a first step.You might check your ThingSpeak credentials using a desktop MQTT Client just to ensure everything works outside of your project itself as a first step.
This example uses MQTT.fx, but I normally use http://mqtt-explorer.com/
We are using the Digi LTE-M/NB IoT platform to automate and recover historical COOP sites that have closed. We send 5 minute temp and 15 minute precip. I don’t have any experience using thingspeak, but we have great success with this platform using the micropython programming option built into the Xbees for MQTT. We use a different arduino based platform for the data acquisition and then have the XBee do the heavy lifting for encryption and TLS. It also has capability for FTPS to push remote updates without the need for the Digi Remote Manager. Very capable device.
Dashboard –https://www.weather.gov/crh/ccoop
-
2021-07-28 at 4:10 PM #15736
In the XBee DE field, type 75B. MQTT port is 1883 (0x75B).
-
2021-07-28 at 4:33 PM #15737Hi @zeke – good check by Rick.
Looking at your error, it does say connection error so implies it didn’t even connect to thingspeak.
I checked this 5months ago and it worked for me. What
Hi @zeke – good check by Rick.Looking at your error, it does say connection error so implies it didn’t even connect to thingspeak.
I checked this 5months ago and it worked for me. What is your channel number?. This allows a view of if any data has arrived. The channel I used for testing, a bit simplistic https://thingspeak.com/channels/5940
You could also enable some debug and share it.
For platformIO this is
<div>
<div>build_flags =</div>
</div>
<div>
<div> -DMS_DATAPUBLISHERBASE_DEBUG</div>
<div> -DMS_THINGSPEAKPUBLISHER_DEBUG</div>
</div>
-
2021-07-28 at 4:47 PM #15738Hi there! Thank you @neilh20 and @vogelrnws for all of your help! Yeah, based on what I’m seeing, I don’t think its been connecting at all. I have been checking the channel I’m usingHi there! Thank you @neilh20 and @vogelrnws for all of your help! Yeah, based on what I’m seeing, I don’t think its been connecting at all. I have been checking the channel I’m using whenever I’m trying to connect and no data has been popping up in the fields. It’s only got like, 5 fields, so it’s not too complicated either. Sorry, I’m somewhat new to PlatformIO, where do I type this stuff?
<div>
<div>build_flags =</div>
</div>
<div>
<div> -DMS_DATAPUBLISHERBASE_DEBUG</div>
<div> -DMS_THINGSPEAKPUBLISHER_DEBUG</div>
</div> -
2021-07-28 at 4:50 PM #15739
Also, I was having trouble finding the stream debugger line (#if defined STREAMDEBUGGER_DBG) you were mentioning earlier in the ModularSensors library. Do you know where that would be located?
-
2021-07-28 at 4:56 PM #15740I’ll have to get about streamdebugger as heading out the office, back in 2hrs
For debug you should see the following in platformIO.ini – add it to the end. It will force a complete recomp
I’ll have to get about streamdebugger as heading out the office, back in 2hrsFor debug you should see the following in platformIO.ini – add it to the end. It will force a complete recompile
<!–more–>
[env:mayfly]
monitor_speed = 115200
board = mayfly
platform = atmelavr
framework = arduino
lib_ldf_mode = deep+
lib_ignore =
RTCZerobuild_flags =
-DSDI12_EXTERNAL_PCINT
-DNEOSWSERIAL_EXTERNAL_PCINT
-DMQTT_MAX_PACKET_SIZE=240
-DTINY_GSM_RX_BUFFER=64
-DTINY_GSM_YIELD_MS=2
-DMS_DATAPUBLISHERBASE_DEBUG
-DMS_THINGSPEAKPUBLISHER_DEBUG -
2021-07-28 at 5:34 PM #15741Hey, thanks @neilh20. I added that and uploaded the program and this was the serial monitor output. What does dataPublisherBase refer to?
Now running logging_to_thingspeak.ino on Logger logger
Using
Hey, thanks @neilh20. I added that and uploaded the program and this was the serial monitor output. What does dataPublisherBase refer to?Now running logging_to_thingspeak.ino on Logger logger
Using ModularSensors Library version 0.28.5
TinyGSM Library version 0.10.9Logger timezone is set to UTC -5
RTC timezone is set to UTC
CTDcond has a non-unique UUID!
CTDtemp has a non-unique UUID!
CTDdepth has a non-unique UUID!
Battery has a non-unique UUID!
12345678-abcd-1234-ef00-1234567890ab -> CTDcond
12345678-abcd-1234-ef00-1234567890ab -> CTDtemp
12345678-abcd-1234-ef00-1234567890ab -> CTDdepth
12345678-abcd-1234-ef00-1234567890ab -> Battery
12345678-abcd-1234-ef00-1234567890ab -> decibelMiliWattCurrent RTC time is: 2021-07-28T16:08:52-05:00
CTDcond has a non-unique UUID!
CTDtemp has a non-unique UUID!
CTDdepth has a non-unique UUID!
Battery has a non-unique UUID!
12345678-abcd-1234-ef00-1234567890ab -> CTDcond
12345678-abcd-1234-ef00-1234567890ab -> CTDtemp
12345678-abcd-1234-ef00-1234567890ab -> CTDdepth
12345678-abcd-1234-ef00-1234567890ab -> Battery
12345678-abcd-1234-ef00-1234567890ab -> decibelMiliWattThis logger has a variable array with 5 variables, of which 4 come from 2 sensors and 1 are calculated.
Logger portion of setup finished.Setting up sensors…
Attempting to connect to the internet and synchronize RTC with NIST
This may take up to two minutes!
Clock already within 5 seconds of time.
Putting modem to sleep
Setting up file on SD card
Data will be saved as logger_2021-07-28.csv
Putting processor to sleep
——————————————\/—- Line Saved to SD Card —-\/
2021-07-28 16:12:00,0.0,22.30,-4.0,4.882,-69Sending data to [ 0 ] mqtt.thingspeak.com
5 fields will be sent to ThingSpeak <–
Topic [ 41 ]: channels/thingSpeakChannelID/publish/thingSpeakChannelKey <–Dumping the TX Buffer <–dataPublisherBase
Message [ 96 ]: created_at=2021-07-28T16:12:00-05:00&field1=0.0&field2=22.30&field3=-4.0&field4=4.882&field5=-69 <–
Opening MQTT Connection <–
MQTT connection failed with state: -4: MQTT_CONNECTION_TIMEOUT
Disconnecting from MQTT <–
Disconnected after 626 ms <–
————————————————————————————
\/—- Line Saved to SD Card —-\/
2021-07-28 16:13:00,0.0,22.30,-3.8,4.867,0And from here it just cycles with the same stuff…
-
2021-07-28 at 11:33 PM #15742Hi @zeke-holloman – looks like the ThingSpeak keys and channel need to be added.
Add them to (the following are examples)
const char* thingSpeakMQTTKey =
“Z0G6QX4IXEHKG0PK”; // YourHi @zeke-holloman – looks like the ThingSpeak keys and channel need to be added.Add them to (the following are examples)
const char* thingSpeakMQTTKey =
“Z0G6QX4IXEHKG0PK”; // Your MQTT API Key from Account > MyProfile.
const char* thingSpeakChannelID =
“5940”; // The numeric channel id for your channel eg
const char* thingSpeakChannelKey =
“DHCKWX0HZQLXGIAX”; // The Write API Key for your channelYour response is
Topic [ 41 ]: channels/thingSpeakChannelID/publish/thingSpeakChannelKeyand it should look something like
Topic [ 38 ]: channels/5940/publish/DHCKWX0HZQLXGIAX <–ThingSpeakPublisher
Followed by this
Dumping the TX Buffer <–dataPublisherBase
Message [ 129 ]: created_at=2021-07-28T19:04:00-08:00&field1=1&field2=2000.000&field3=-0.223&field4=4.1748&field5=4.215&field6=22.20&field7=0.0088 <–ThingSpeakPublisher
Opening MQTT Connection <–ThingSpeakPublisher
MQTT connected after 874 ms <–ThingSpeakPublisher
ThingSpeak topic published! Current state: 0: MQTT_CONNECTED
Disconnecting from MQTT <–ThingSpeakPublisher
Disconnected after 418 ms <–ThingSpeakPublisher -
2021-07-29 at 8:47 AM #15744Hey @neilh20. Thanks for looking at. Yeah, I already had the keys and the channel ID in the code, I just took them out for the forum. Sorry, I understand that could have been confusing. Other than thaHey @neilh20. Thanks for looking at. Yeah, I already had the keys and the channel ID in the code, I just took them out for the forum. Sorry, I understand that could have been confusing. Other than that, is there anything else you could see it being?
-
2021-07-29 at 11:24 AM #15751@zeke-holloman
I’m not sure if this pertains to your project or not, but when I was looking through the ThingSpeak documentation and going through my test account it had a place to add a MQTT d
@zeke-hollomanI’m not sure if this pertains to your project or not, but when I was looking through the ThingSpeak documentation and going through my test account it had a place to add a MQTT device that would be posting. It issued the device credentials: username, client ID and password…?? Sorry if I’m just causing more confusion, but trying to see what is keeping your unit from the initial broker connection
https://www.mathworks.com/help/thingspeak/mqtt-basics.html#mw_0bf68abc-6c4e-4e2e-8312-e8223a203b71
-
2021-07-29 at 11:41 AM #15753@vogelrnws
Hi there! Yeah I registered the MQTT device via ThingSpeak. I used the MQTT Explorer you suggested and I was able to post some data to Thingspeak by typing literal values into the
@vogelrnwsHi there! Yeah I registered the MQTT device via ThingSpeak. I used the MQTT Explorer you suggested and I was able to post some data to Thingspeak by typing literal values into the Explorer. This leads me to think that it’s an issue with the Mayfly code.
@neilh20 I’m curious as to how you set up the ThingSpeak using POST. I’m not really committed to the MQTT platform, and I’m curious how you set your’s up. -
2021-07-29 at 12:22 PM #15754@zeke-holloman a correction on my part, when I looked through the thingspeak API, both the historical and the MQTT method use the POST@zeke-holloman a correction on my part, when I looked through the thingspeak API, both the historical and the MQTT method use the POST to deliver it to thingspeak, however they have a slightly different API.
Initially when looking at your posting I assumed you had removed the keys as it was public, however looking at your your trace it is saying that it doesn’t have the keys in it.
Your response is
Topic [ 41 ]: channels/thingSpeakChannelID/publish/thingSpeakChannelKeyand it should look something like
Topic [ 38 ]: channels/5940/publish/DHCKWX0HZQLXGIAX <–ThingSpeakPublisher
Please confirm that your run has the API keys in.
If that doesn’t work, can you post your platformio.ini
I’m committed to something for the next couple of hours but I can add describe the STREAMDEBUG setup after looking at your platformio.ini
-
2021-07-29 at 12:37 PM #15755Hi @neilh20, thanks for all your help! Yeah, I think I replaced the channel key in the trace with the thingSpeakChannelKey for the forum. I’m looking at the Topic [41] thing and I see the key inHi @neilh20, thanks for all your help! Yeah, I think I replaced the channel key in the trace with the thingSpeakChannelKey for the forum. I’m looking at the Topic [41] thing and I see the key in the topic channels/…
Here’s the platformIO file.
[platformio]
src_dir = src/logging_to_thingspeak
;src_dir = src/LTExBee_FirstConnectiondescription = ModularSensors example intended for DRWI users with CTD, turbidity, and 2G signal
[env:mayfly]
monitor_speed = 115200
board = mayfly
platform = atmelavr
framework = arduino
lib_ldf_mode = deep+
lib_ignore =
RTCZero
Adafruit NeoPixel
Adafruit GFX Library
Adafruit SSD1306
Adafruit ADXL343
Adafruit STMPE610
Adafruit TouchScreen
Adafruit ILI9341
build_flags =
-DSDI12_EXTERNAL_PCINT
-DNEOSWSERIAL_EXTERNAL_PCINT
-DMQTT_MAX_PACKET_SIZE=240
-DTINY_GSM_RX_BUFFER=64
-DTINY_GSM_YIELD_MS=2
-DMS_DATAPUBLISHERBASE_DEBUG
-DMS_THINGSPEAKPUBLISHER_DEBUG
lib_deps =
envirodiy/EnviroDIY_ModularSensors -
2021-07-29 at 6:18 PM #15756
@zeke-holloman interesting, seems like it should work.
When I have strange issues, I delete the .pio/… and force it to do a refresh with all the remote libs. I see you a
@zeke-holloman interesting, seems like it should work.
When I have strange issues, I delete the .pio/… and force it to do a refresh with all the remote libs. I see you are using the DecagonCTD which recently changed to hydros21, but seems like it should work.
I’ve taken your platformio.ini and program and adding my thingspeak keys, and will try running. I have a test system with Digi LTE I’m in the process of switching so I can try your setup
In the meantime
platformio.ini to
build_flags = -DSTREAMDEBUGGER_DBG #usually add at end of list
lib_deps =
https://github.com/vshymanskyy/StreamDebugger ;Debug when neededlogging_to_thinkgspeak : look for DigiXBeeCellularTransparent and make changes
// Create the modem object
<span style=”color: #ff0000;”>#if defined STREAMDEBUGGER_DBG</span>
<span style=”color: #ff0000;”>#include <StreamDebugger.h></span>
<span style=”color: #ff0000;”>StreamDebugger modemDebugger(modemSerial, STANDARD_SERIAL_OUTPUT);</span>
<span style=”color: #ff0000;”>#define modemSerHw modemDebugger</span>
<span style=”color: #ff0000;”>#else</span>
<span style=”color: #ff0000;”>#define modemSerHw modemSerial</span>
<span style=”color: #ff0000;”>#endif // STREAMDEBUGGER_DBG</span>
DigiXBeeCellularTransparent modemXBCT(&<span style=”color: #ff0000;”>modemSerHw</span>, modemVccPin, modemStatusPin,
useCTSforStatus, modemResetPin,
modemSleepRqPin, apn); -
2021-07-29 at 8:07 PM #15757@zeke-holloman – I have no idea why my previous post came out so badly, and it won’t allow me to edit it!!. My usual way of recovering strange posts.
It should be interesting to see what
@zeke-holloman – I have no idea why my previous post came out so badly, and it won’t allow me to edit it!!. My usual way of recovering strange posts.It should be interesting to see what you get with your cell modem.
Using your sketch, which references the latest ModularSensors
Using ModularSensors Library version 0.30.0
TinyGSM Library version 0.11.4My cell modem which is running on Verizon wouldn’t connect. The modem issues are technical and I believe understood, and have been discussed.
This specific modem has been working no problem in my fork which has modifications for managing the startup of the cell modem and works for the cell company I’ve been using …. my fork currently is at ModularSensors 0.28.5
I did try compiling your program in my fork, but there is some changes in 0.30.0 that aren’t backward compatible, so I need to upgrade before I can try this.
Still with your software, referncing MS 0.30.0 I switched the modem to Digi WiFi S6B, which I use a lot in testing, and it worked no problem.
Once connected to an TCP/IP link – whether over cell or wifi – the passage of data is the same, though Modem AT cmds may be a little different.
So this is what I get
123456789101112131415161718192021222324252627282930Sending data to [ 0 ] mqtt.thingspeak.com5 fields will be sent to ThingSpeak <--ThingSpeakPublisherTopic [ 38 ]: channels/5940/publish/DHCKWX0HZQLHGIAX <--ThingSpeakPublisherDumping the TX Buffer <--dataPublisherBaseMessage [ 106 ]: created_at=2021-07-29T18:34:00-05:00&field1=-9999.0&field2=-9999.00&field3=-9999.0&field4=4.761&field5=-25 <--ThingSpeakPublisherOpening MQTT Connection <--ThingSpeakPublisher+++OKATLAmqtt.thingspeak.com34.194.225.123ATIP1ATDL132.163.97.6ATDL34.194.225.123OKATDE25ATDE75bOKATWROKATACOKATCNOK$MQTTÂMSMSZ0G6QS4IXEHKG0PK MQTT connected after 838 ms <--ThingSpeakPublisher0&channels/5940/publish/DHCKWX0HZQLHGIAXcreated_at=2021-07-29T18:34:00-05:00&field1=-9999.0&field2=-9999.00&field3=-9999.0&field4=4.761&field5=-25ThingSpeak topic published! Current state: 0: MQTT_CONNECTEDDisconnecting from MQTT <--ThingSpeakPublisherà+++OKATTM -
2021-08-02 at 1:32 PM #15759Hi there @neilh20. Thanks for your response, sorry I couldn’t get to it until now. I spent some time on the code, and I tried the code with the 0.28.5 version of the EnviroDIY ModularSensors libHi there @neilh20. Thanks for your response, sorry I couldn’t get to it until now. I spent some time on the code, and I tried the code with the 0.28.5 version of the EnviroDIY ModularSensors library just to see if anything changed. It still isn’t uploading to ThingSpeak though. I tried the 0.30.0 version of the ModularSensors library, alongside the StreamDebugger as well, here’s the output of the code (excluding the ThingSpeak data). It doesn’t look that much different.
Windows PowerShell
Copyright (C) Microsoft Corporation. All rights reserved.Try the new cross-platform PowerShell https://aka.ms/pscore6
PS C:\Users\waked\OneDrive – East Carolina University\Documents\PlatformIO\Projects\LearnEnviroDIYcode-master> pio device monitor –port COM10 –baud 115200
— Available filters and text transformations: colorize, debug, default, direct, hexlify, log2file, nocontrol, printable, send_on_enter, time
— More details at http://bit.ly/pio-monitor-filters
— Miniterm on COM10 115200,8,N,1 —
— Quit: Ctrl+C | Menu: Ctrl+T | Help: Ctrl+T followed by Ctrl+H —
Now running logging_to_thingspeak5.ino on Logger loggerUsing ModularSensors Library version TinyGSM Library version 0.11.4
Current RTC time is: 2021-08-02T12:16:53-05:00
CTDcond has a non-unique UUID!
CTDtemp has a non-unique UUID!
CTDdepth has a non-unique UUID!
Battery has a non-unique UUID!
12345678-abcd-1234-ef00-1234567890ab -> CTDcond
12345678-abcd-1234-ef00-1234567890ab -> CTDtemp
12345678-abcd-1234-ef00-1234567890ab -> CTDdepth
12345678-abcd-1234-ef00-1234567890ab -> Battery
12345678-abcd-1234-ef00-1234567890ab -> decibelMiliWattThis logger has a variable array with 5 variables, of which 4 come from 2 sensors and 1 are calculated.
Setting up sensors…
Attempting to connect to the internet and synchronize RTC with NIST
This may take up to two minutes!
OKCN163.97.697.6
OKCN163.97.6
OKCN163.97.6
OKCN163.97.6
OKCN163.97.6Clock already within 5 seconds of time.
Putting modem to sleep
Setting up file on SD card
Data will be saved as logger_2021-08-02.csv
Putting processor to sleep
——————————————\/—- Line Saved to SD Card —-\/
2021-08-02 12:20:00,0.0,22.40,-3.2,4.897,-81OKCN3.23.121
Sending data to [ 0 ] mqtt.thingspeak.com
MQTT connection failed with state: -4: MQTT_CONNECTION_TIMEOUT——————————————
——————————————
\/—- Line Saved to SD Card —-\/
2021-08-02 12:21:00,0.0,22.40,-2.7,4.897,0OKCN3.23.121
Sending data to [ 0 ] mqtt.thingspeak.com
MQTT connection failed with state: -4: MQTT_CONNECTION_TIMEOUTAdditionally, here’s the updated version of the code with the StreamDebugger stuff.
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356/** =========================================================================* @file logging_to_ThingSpeak.ino* @brief Example logging data and publishing to ThingSpeak.** @author Sara Geleskie Damiano <sdamiano@stroudcenter.org>* @copyright (c) 2017-2020 Stroud Water Research Center (SWRC)* and the EnviroDIY Development Team* This example is published under the BSD-3 license.** Build Environment: Visual Studios Code with PlatformIO* Hardware Platform: EnviroDIY Mayfly Arduino Datalogger** DISCLAIMER:* THIS CODE IS PROVIDED "AS IS" - NO WARRANTY IS GIVEN.* ======================================================================= */// ==========================================================================// Defines for the Arduino IDE// NOTE: These are ONLY needed to compile with the Arduino IDE.// If you use PlatformIO, you should set these build flags in your// platformio.ini// ==========================================================================/** Start [defines] */#ifndef TINY_GSM_RX_BUFFER#define TINY_GSM_RX_BUFFER 64#endif#ifndef TINY_GSM_YIELD_MS#define TINY_GSM_YIELD_MS 2#endif#ifndef MQTT_MAX_PACKET_SIZE#define MQTT_MAX_PACKET_SIZE 240#endif#define TINY_GSM_MODEM_XBEE/** End [defines] */// ==========================================================================// Include the libraries required for any data logger// ==========================================================================/** Start [includes] */// The Arduino library is needed for every Arduino program.#include <Arduino.h>// EnableInterrupt is used by ModularSensors for external and pin change// interrupts and must be explicitly included in the main program.#include <EnableInterrupt.h>// To get all of the base classes for ModularSensors, include LoggerBase.// NOTE: Individual sensor definitions must be included separately.#include <LoggerBase.h>#include <StreamDebugger.h>/** End [includes] */// ==========================================================================// Data Logging Options// ==========================================================================/** Start [logging_options] */// The name of this program fileconst char* sketchName = "logging_to_thingspeak5.ino";// Logger ID, also becomes the prefix for the name of the data file on SD cardconst char* LoggerID = "logger";// How frequently (in minutes) to log dataconst uint8_t loggingInterval = 1;// Your logger's timezone.const int8_t timeZone = -5; // Eastern Standard Time// NOTE: Daylight savings time will not be applied! Please use standard time!// Set the input and output pins for the logger// NOTE: Use -1 for pins that do not applyconst int32_t serialBaud = 115200; // Baud rate for debuggingconst int8_t greenLED = 8; // Pin for the green LEDconst int8_t redLED = 9; // Pin for the red LEDconst int8_t buttonPin = 21; // Pin for debugging mode (ie, button pin)const int8_t wakePin = A7; // MCU interrupt/alarm pin to wake from sleep// Set the wake pin to -1 if you do not want the main processor to sleep.// In a SAMD system where you are using the built-in rtc, set wakePin to 1const int8_t sdCardPwrPin = -1; // MCU SD card power pinconst int8_t sdCardSSPin = 12; // SD card chip select/slave select pinconst int8_t sensorPowerPin = 22; // MCU pin controlling main sensor power/** End [logging_options] */// ==========================================================================// Wifi/Cellular Modem Options// ==========================================================================/** Start [xbee_cell_transparent] */// For any Digi Cellular XBee's// NOTE: The u-blox based Digi XBee's (3G global and LTE-M global) can be used// in either bypass or transparent mode, each with pros and cons// The Telit based Digi XBees (LTE Cat1) can only use this mode.#if defined STREAMDEBUGGER_DBG#include <StreamDebugger.h>#include <modems/DigiXBeeCellularTransparent.h>// NOTE: Extra hardware and software serial ports are created in the "Settings// for Additional Serial Ports" sectionHardwareSerial& modemSerial = Serial1; // Use hardware serial if possibleconst int32_t modemBaud = 9600; // All XBee's use 9600 by defaultStreamDebugger modemDebugger(modemSerial, STANDARD_SERIAL_OUTPUT);#define modemSerHw modemDebugger#else#define modemSerHw modemSerial#endif // STREAMDEBUGGER_DBG// Modem Pins - Describe the physical pin connection of your modem to your board// NOTE: Use -1 for pins that do not apply// The pin numbers here are for a Digi XBee with a Mayfly and LTE adapter// For options https://github.com/EnviroDIY/LTEbee-Adapter/edit/master/README.mdconst int8_t modemVccPin = -1; // MCU pin controlling modem power// Option: modemVccPin = A5, if Mayfly SJ7 is// connected to the ASSOC pinconst int8_t modemStatusPin = 13; // MCU pin used to read modem status// NOTE: If possible, use the <code>STATUS/SLEEP_not</code> (XBee pin 13) for status, but// the CTS pin can also be used if necessaryconst bool useCTSforStatus = false; // Flag to use the CTS pin for statusconst int8_t modemResetPin = 20; // MCU pin connected to modem reset pinconst int8_t modemSleepRqPin = 23; // MCU pin for modem sleep/wake requestconst int8_t modemLEDPin = redLED; // MCU pin connected an LED to show modem// status// Network connection informationconst char* apn = "hologram"; // APN for GPRS connection// Create the modem objectDigiXBeeCellularTransparent modemXBCT(&modemSerHw, modemVccPin, modemStatusPin,useCTSforStatus, modemResetPin,modemSleepRqPin, apn);// Create an extra reference to the modem by a generic nameDigiXBeeCellularTransparent modem = modemXBCT;/** End [xbee_cell_transparent] */// ==========================================================================// ==========================================================================// Using the Processor as a Sensor// ==========================================================================/** Start [processor_sensor] */#include <sensors/ProcessorStats.h>// Create the main processor chip "sensor" - for general metadataconst char* mcuBoardVersion = "v0.5b";ProcessorStats mcuBoard(mcuBoardVersion);/** End [processor_sensor] */// ==========================================================================// Maxim DS3231 RTC (Real Time Clock)// ==========================================================================/** Start [ds3231] */#include <sensors/MaximDS3231.h>// Create a DS3231 sensor objectMaximDS3231 ds3231(1);/** End [ds3231] */// ==========================================================================// Meter Hydros 21 Conductivity, Temperature, and Depth Sensor// ==========================================================================/** Start [decagon_ctd] */#include <sensors/DecagonCTD.h>const char* CTDSDI12address = "1"; // The SDI-12 Address of the CTDconst uint8_t CTDNumberReadings = 6; // The number of readings to averageconst int8_t SDI12Power = sensorPowerPin; // Power pin (-1 if unconnected)const int8_t SDI12Data = 7; // The SDI12 data pin// Create a Decagon CTD sensor objectDecagonCTD ctd(*CTDSDI12address, SDI12Power, SDI12Data, CTDNumberReadings);/** End [decagon_ctd] */// ==========================================================================// Creating the Variable Array[s] and Filling with Variable Objects// ==========================================================================/** Start [variable_arrays] */Variable* variableList[] = {new DecagonCTD_Cond(&ctd, "12345678-abcd-1234-ef00-1234567890ab"),new DecagonCTD_Temp(&ctd, "12345678-abcd-1234-ef00-1234567890ab"),new DecagonCTD_Depth(&ctd, "12345678-abcd-1234-ef00-1234567890ab"),new ProcessorStats_Battery(&mcuBoard,"12345678-abcd-1234-ef00-1234567890ab"),new Modem_RSSI(&modem, "12345678-abcd-1234-ef00-1234567890ab")};// Count up the number of pointers in the arrayint variableCount = sizeof(variableList) / sizeof(variableList[0]);// Create the VariableArray objectVariableArray varArray;/** End [variable_arrays] */// ==========================================================================// The Logger Object[s]// ==========================================================================/** Start [loggers] */// Create a logger instanceLogger dataLogger;/** End [loggers] */// ==========================================================================// Creating Data Publisher[s]// ==========================================================================// Create a channel with fields on ThingSpeak in advance// The fields will be sent in exactly the order they are in the variable array.// Any custom name or identifier given to the field on ThingSpeak is irrelevant.// No more than 8 fields of data can go to any one channel. Any fields beyond// the eighth in the array will be ignored.const char* thingSpeakMQTTKey ="thingSpeakMQTTKey"; // Your MQTT API Key from Account > MyProfile.const char* thingSpeakChannelID ="thingSpeakChannelID"; // The numeric channel id for your channelconst char* thingSpeakChannelKey ="thingSpeakChannelKey"; // The Write API Key for your channel// Create a data publisher for ThingSpeak#include <publishers/ThingSpeakPublisher.h>ThingSpeakPublisher TsMqtt;/** End [loggers] */// ==========================================================================// Working Functions// ==========================================================================/** Start [working_functions] */// Flashes the LED's on the primary boardvoid greenredflash(uint8_t numFlash = 4, uint8_t rate = 75) {for (uint8_t i = 0; i < numFlash; i++) {digitalWrite(greenLED, HIGH);digitalWrite(redLED, LOW);delay(rate);digitalWrite(greenLED, LOW);digitalWrite(redLED, HIGH);delay(rate);}digitalWrite(redLED, LOW);}// Reads the battery voltage// NOTE: This will actually return the battery level from the previous update!float getBatteryVoltage() {if (mcuBoard.sensorValues[0] == -9999) mcuBoard.update();return mcuBoard.sensorValues[0];}/** End [working_functions] */// ==========================================================================// Arduino Setup Function// ==========================================================================/** Start [setup] */void setup() {// Start the primary serial connectionSerial.begin(serialBaud);// Print a start-up note to the first serial portSerial.print(F("Now running "));Serial.print(sketchName);Serial.print(F(" on Logger "));Serial.println(LoggerID);Serial.println();Serial.print(F("Using ModularSensors Library version "));//Serial.println(MODULAR_SENSORS_VERSION);Serial.print(F("TinyGSM Library version "));Serial.println(TINYGSM_VERSION);Serial.println();// Start the serial connection with the modemmodemSerial.begin(modemBaud);// Set up pins for the LED'spinMode(greenLED, OUTPUT);digitalWrite(greenLED, LOW);pinMode(redLED, OUTPUT);digitalWrite(redLED, LOW);// Blink the LEDs to show the board is on and starting upgreenredflash();// Set the timezones for the logger/data and the RTC// Logging in the given time zoneLogger::setLoggerTimeZone(timeZone);// It is STRONGLY RECOMMENDED that you set the RTC to be in UTC (UTC+0)Logger::setRTCTimeZone(0);// Attach the modem and information pins to the loggerdataLogger.attachModem(modem);modem.setModemLED(modemLEDPin);dataLogger.setLoggerPins(wakePin, sdCardSSPin, sdCardPwrPin, buttonPin,greenLED);// Begin the variable array[s], logger[s], and publisher[s]varArray.begin(variableCount, variableList);dataLogger.begin(LoggerID, loggingInterval, &varArray);TsMqtt.begin(dataLogger, &modem.gsmClient, thingSpeakMQTTKey,thingSpeakChannelID, thingSpeakChannelKey);// Note: Please change these battery voltages to match your battery// Set up the sensors, except at lowest battery levelif (getBatteryVoltage() > 3.4) {Serial.println(F("Setting up sensors..."));varArray.setupSensors();}// Sync the clock if it isn't valid or we have battery to spareif (getBatteryVoltage() > 3.55 || !dataLogger.isRTCSane()) {// Synchronize the RTC with NIST// This will also set up the modemdataLogger.syncRTC();}// Create the log file, adding the default header to it// Do this last so we have the best chance of getting the time correct and// all sensor names correct// Writing to the SD card can be power intensive, so if we're skipping// the sensor setup we'll skip this too.if (getBatteryVoltage() > 3.4) {Serial.println(F("Setting up file on SD card"));dataLogger.turnOnSDcard(true); // true = wait for card to settle after power updataLogger.createLogFile(true); // true = write a new headerdataLogger.turnOffSDcard(true); // true = wait for internal housekeeping after write}// Call the processor sleepSerial.println(F("Putting processor to sleep"));dataLogger.systemSleep();}/** End [setup] */// ==========================================================================// Arduino Loop Function// ==========================================================================/** Start [loop] */// Use this short loop for simple data logging and sendingvoid loop() {// Note: Please change these battery voltages to match your battery// At very low battery, just go back to sleepif (getBatteryVoltage() < 3.4) {dataLogger.systemSleep();}// At moderate voltage, log data but don't send it over the modemelse if (getBatteryVoltage() < 3.55) {dataLogger.logData();}// If the battery is good, send the data to the worldelse {dataLogger.logDataAndPublish();}}/** End [loop] */That’s very strange that the code worked with the Digi Wifi module. Any other suggestions are greatly appreciated! And if you see anything that I did wrong with the code, please let me know! I’m still rather new to this.
-
2021-08-02 at 3:53 PM #15761
Hey @neilh20. I just had a thought. The fact that the software works with the Digi WiFi S6B. Do you think that means its an issue with the actual cellular connection as opposed to the code?
-
2021-08-02 at 4:13 PM #15762hello @zeke-holloman. Just looking through it, and yes thats my assumption at the moment is that its the cellular side that is not connecting.
There are 2 parts 1) getting internet & time 2) thinghello @zeke-holloman. Just looking through it, and yes thats my assumption at the moment is that its the cellular side that is not connecting.
There are 2 parts 1) getting internet & time 2) thingspeak.
1) If the cell phone connects for NIST time, then it can connect with the internet, and the cell modem account setting is good.2) if thingspeak then doesn’t connect its something to do with the account ino.
I’m just looking at your posted new file
The STREAMDEBUGGER_DBG is a bit strange, it should really be this
12345678910111213141516171819202122232425// ==========================================================================// Wifi/Cellular Modem Options// ==========================================================================// NOTE: Extra hardware and software serial ports are created in the "Settings// for Additional Serial Ports" sectionHardwareSerial& modemSerial = Serial1; // Use hardware serial if possible#if defined STREAMDEBUGGER_DBG#include <StreamDebugger.h>StreamDebugger modemDebugger(modemSerial, STANDARD_SERIAL_OUTPUT);#define modemSerHw modemDebugger#else#define modemSerHw modemSerial#endif // STREAMDEBUGGER_DBG//#define DigiXBeeCellularTransparent_Module#define DigiXBeeWifi_Module#ifdef DigiXBeeCellularTransparent_Module/** Start [xbee_cell_transparent] */// For any Digi Cellular XBee's// NOTE: The u-blox based Digi XBee's (3G global and LTE-M global) can be used// in either bypass or transparent mode, each with pros and cons// The Telit based Digi XBees (LTE Cat1) can only use this mode.#include <modems/DigiXBeeCellularTransparent.h>const int32_t modemBaud = 9600; // All XBee's use 9600 by defaultThe log you posted with some basic cell phone interface is looking very minimal. Possibly also compile it with
-DMS_DATAPUBLISHERBASE_DEBUG
-DMS_THINGSPEAKPUBLISHER_DEBUG-DSTREAMDEBUGGER_DBG
.
I was thinking of creating a branch off envirodiy (master) – and seeing if it can be referenced for an update to the cell phone interface so it can show more detail. Gotta try it out. -
2021-08-02 at 4:28 PM #15763@neilh20 and @zeke-holloman
Gonna let you all work through this on the mayfly code using the xbee transparent mode. If at some point you would like to use the micro-python option in the xbee I can pr
@neilh20 and @zeke-hollomanGonna let you all work through this on the mayfly code using the xbee transparent mode. If at some point you would like to use the micro-python option in the xbee I can provide some code to just use simple serial.println() on the mayfly to send payload to the xbee and it will handle all the MQTT stuff in the background.
-
2021-08-02 at 5:23 PM #15764Hi there @vogelrnws. Honestly, It’d be awesome if you could send that code. I’m still considering my options on the method to publish the data, but that could be helpful!Hi there @vogelrnws. Honestly, It’d be awesome if you could send that code. I’m still considering my options on the method to publish the data, but that could be helpful!
@neilh20, ok! I’ll try that. For some reason (not really sure why), the code all of a sudden started saying that it couldn’t connect to the internet for clock sync. I’m looking into that, so I’ll get back to you once I fix that. Thank you again! -
2021-08-02 at 5:46 PM #15765Hello @rick-vogel – sounds fascinating & fun to use python in the Xbee. Though for ModularSensors there are layers of software that allow the physical modem to be abstracted so it could beHello @rick-vogel – sounds fascinating & fun to use python in the Xbee. Though for ModularSensors there are layers of software that allow the physical modem to be abstracted so it could be possible to use another LTE when it becomes available (0.30.0 seems to have hooks for a new LTE modem SIM7080G) – through TinyGSM. As a Class package that @srgdamiano has put together its quite comprehensive – though as with all abstractions it has upsides and downsides. I think this is working through what it takes to figure out the control flow, which would be needed whatever route is taken. In my fork (with some LTE interface changes) it has worked.
@zeke-holloman, I’d got pulled off onto something else, now I’ve added the STREAMDEBUGGER_DBG to your “Logging_to_Thingskeak5.ino”and removed my extra code that I had and about to test it – but I have to get another dataplan going to test it – I use https://dataplans.digikey.com/ that has a 50Mbytes/month plans,1234567891011121314151617181920212223242526272829303132333435363738394041424344454647/ ==========================================================================// Wifi/Cellular Modem Options// ==========================================================================/** Start [xbee_cell_transparent] */// For any Digi Cellular XBee's// NOTE: The u-blox based Digi XBee's (3G global and LTE-M global) can be used// in either bypass or transparent mode, each with pros and cons// The Telit based Digi XBees (LTE Cat1) can only use this mode.HardwareSerial& modemSerial = Serial1; // Use hardware serial if possible#if defined STREAMDEBUGGER_DBG#include <StreamDebugger.h>StreamDebugger modemDebugger(modemSerial, STANDARD_SERIAL_OUTPUT);#define modemSerHw modemDebugger#else#define modemSerHw modemSerial#endif // STREAMDEBUGGER_DBG#include <modems/DigiXBeeCellularTransparent.h>const int32_t modemBaud = 9600; // All XBee's use 9600 by default// Modem Pins - Describe the physical pin connection of your modem to your board// NOTE: Use -1 for pins that do not apply// The pin numbers here are for a Digi XBee with a Mayfly and LTE adapter// For options https://github.com/EnviroDIY/LTEbee-Adapter/edit/master/README.mdconst int8_t modemVccPin = -1; // MCU pin controlling modem power// Option: modemVccPin = A5, if Mayfly SJ7 is// connected to the ASSOC pinconst int8_t modemStatusPin = 13; // MCU pin used to read modem status// NOTE: If possible, use the <code>STATUS/SLEEP_not</code> (XBee pin 13) for status, but// the CTS pin can also be used if necessaryconst bool useCTSforStatus = false; // Flag to use the CTS pin for statusconst int8_t modemResetPin = 20; // MCU pin connected to modem reset pinconst int8_t modemSleepRqPin = 23; // MCU pin for modem sleep/wake requestconst int8_t modemLEDPin = redLED; // MCU pin connected an LED to show modem// status// Network connection informationconst char* apn = "hologram"; // APN for GPRS connection// Create the modem objectDigiXBeeCellularTransparent modemXBCT(&modemSerHw, modemVccPin, modemStatusPin,useCTSforStatus, modemResetPin,modemSleepRqPin, apn);// Create an extra reference to the modem by a generic nameDigiXBeeCellularTransparent modem = modemXBCT;/** End [xbee_cell_transparent] */ -
2021-08-02 at 6:33 PM #15766@zeke-holloman, I figured out that the other day I hadn’t changed the apn to the source I was using.
So with your code, my apn a
@zeke-holloman, I figured out that the other day I hadn’t changed the apn to the source I was using.So with your code, my apn and thingspeak API, using tera-term 4.105 with TimeStamp and the following changes I got a good response from thingspeak
#include <ModularSensors.h> //pre 0.30.0 was <LoggerBase.h></div>
The core thingspeak looks like this
1234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950515253545556575859606162636465666768697071727374757677787980818283848586878889nh: it got time OK[2021-08-02 15:16:04.029] Setting up file on SD cardnh: zzz sleeping[2021-08-02 15:16:04.173] Data will be saved as logger_2021-08-02.csv[2021-08-02 15:16:04.198] Putting processor to sleep[2021-08-02 15:17:00.586] ------------------------------------------[2021-08-02 15:17:14.269][2021-08-02 15:17:14.269] \/---- Line Saved to SD Card ----\/[2021-08-02 15:17:14.269] 2021-08-02 17:17:00,-9999.0,-9999.00,-9999.0,4.548,-81[2021-08-02 15:17:14.288][2021-08-02 15:17:14.288][2021-08-02 15:17:14.444] +++OK[2021-08-02 15:17:14.557] ATCN[2021-08-02 15:17:14.557] OK[2021-08-02 15:17:14.932] +++OK[2021-08-02 15:17:15.052] ATAP[2021-08-02 15:17:15.073] 0[2021-08-02 15:17:15.073] ATGT[2021-08-02 15:17:15.073] 64[2021-08-02 15:17:15.073] ATCT[2021-08-02 15:17:15.076] 64[2021-08-02 15:17:15.076] ATHS[2021-08-02 15:17:15.076] B08[2021-08-02 15:17:15.076] ATCN[2021-08-02 15:17:15.092] OK[2021-08-02 15:17:15.224] +++OK[2021-08-02 15:17:15.324] ATAI[2021-08-02 15:17:15.355] 0[2021-08-02 15:17:15.355] ATMY[2021-08-02 15:17:15.355] 100.111.10.79[2021-08-02 15:17:15.371] ATCN[2021-08-02 15:17:15.430] OK[2021-08-02 15:17:15.430] Sending data to [ 0 ] mqtt.thingspeak.com[2021-08-02 15:17:15.430] 5 fields will be sent to ThingSpeak <--ThingSpeakPublisher[2021-08-02 15:17:15.430] Topic [ 38 ]: channels/5940/publish/DHCKWXXHZQLHGIAX <--ThingSpeakPublisher[2021-08-02 15:17:15.430] Dumping the TX Buffer <--dataPublisherBase[2021-08-02 15:17:15.430] Message [ 106 ]: created_at=2021-08-02T17:17:00-05:00&field1=-9999.0&field2=-9999.00&field3=-9999.0&field4=4.548&field5=-81 <--ThingSpeakPublisher[2021-08-02 15:17:15.430] Opening MQTT Connection <--ThingSpeakPublisher[2021-08-02 15:17:15.524] +++OK[2021-08-02 15:17:15.640] ATLAmqtt.thingspeak.com[2021-08-02 15:17:17.797] 3.83.181.186[2021-08-02 15:17:17.812] ATIP[2021-08-02 15:17:17.812] 1[2021-08-02 15:17:17.812] ATDL[2021-08-02 15:17:17.828] 132.163.97.6[2021-08-02 15:17:17.844] ATDL3.83.181.186[2021-08-02 15:17:17.859] OK[2021-08-02 15:17:17.859] ATDE[2021-08-02 15:17:17.881] 25[2021-08-02 15:17:17.881] ATDE75b[2021-08-02 15:17:17.897] OK[2021-08-02 15:17:17.897] ATWR[2021-08-02 15:17:17.952] OK[2021-08-02 15:17:17.952] ATAC[2021-08-02 15:17:17.972] OK[2021-08-02 15:17:17.972] ATCI[2021-08-02 15:17:17.981] FF[2021-08-02 15:17:17.981] ATCN[2021-08-02 15:17:17.999] OK[2021-08-02 15:17:17.999] $MQTTÂMSMSZXG6QS4IXEHKG0PK+++OK[2021-08-02 15:17:18.260] ATCI[2021-08-02 15:17:18.282] FF[2021-08-02 15:17:18.282] ATOD[2021-08-02 15:17:18.282] 0.0.0.0[2021-08-02 15:17:18.298] ATCN[2021-08-02 15:17:18.298] OK[2021-08-02 15:17:19.898] MQTT connected after 4483 ms <--ThingSpeakPublisher[2021-08-02 15:17:19.921] 0&channels/5940/publish/DHCKWX0HZQLHGIAXcreated_at=2021-08-02T17:17:00-05:00&field1=-9999.0&field2=-9999.00&field3=-9999.0&field4=4.548&field5=-81+++OK[2021-08-02 15:17:20.304] ATCI[2021-08-02 15:17:20.304] 0[2021-08-02 15:17:20.304] ATOD[2021-08-02 15:17:20.324] 3.83.181.186[2021-08-02 15:17:20.324] ATCN[2021-08-02 15:17:20.338] OK[2021-08-02 15:17:20.338] ThingSpeak topic published! Current state: 0: MQTT_CONNECTED[2021-08-02 15:17:20.357] Disconnecting from MQTT <--ThingSpeakPublisher[2021-08-02 15:17:20.357] à+++OK[2021-08-02 15:17:20.581] ATTM[2021-08-02 15:17:20.597] 64[2021-08-02 15:17:20.597] ATTM64[2021-08-02 15:17:20.613] OK[2021-08-02 15:17:20.613] ATWR[2021-08-02 15:17:20.661] OK[2021-08-02 15:17:20.661] ATAC[2021-08-02 15:17:20.677] OK[2021-08-02 15:17:20.677] ATCN[2021-08-02 15:17:20.691] OK[2021-08-02 15:17:20.691] Disconnected after 347 ms <--ThingSpeakPublisher[2021-08-02 15:17:20.818] +++OKI have a lot more debug, and better scale-ability built into my fork https://github.com/neilh10/ModularSensors but the above worked for me.
-
2021-08-02 at 6:42 PM #15767
https://thingspeak.com/channels/5940 shows a data point at Aug 02 2021 15:17 GMT-0700 Yeah!!
-
2021-08-03 at 7:38 AM #15768Here are the python files if you ever decide to look at that option. Please feel free to manipulate however you need to.
- Open up the main.py file in a ‘python’ editor and use th
Here are the python files if you ever decide to look at that option. Please feel free to manipulate however you need to.- Open up the main.py file in a ‘python’ editor and use the credentials you get from the MQTT device you created on thingspeak, not your API keys. Save file.
- Within Digi XTCU you will need to go to File System Manager on the Xbee and go into the “lib” folder.
- Create a folder named umqtt. (right click, create directory, name it, enter) Place the simple.py file in that folder.
- Go back to the main /flash directory (should show cert and lib folders) and transfer the main.py file into that location.
This main file does a few different things. Sorry, I didn’t want to completely rewrite the file so I used a lot of what I already had in place and deleted the FTPS portions.
- It will recognize data that it needs to publish from the mayfly be adding 2 things.
- #P on the front of the data string (no space between this and the actual data)
- ‘\n’ on the end of the data string (println() from mayfly for each data string you want to send works)
- so….. #Pfield1=xx&field2=xx&field3=xx……\r\n
- python uses the \n to separate the strings and then trims the \r later.
- you can send multiple strings at once as long as each one has these two identifiers on the front and back and this file will split them into individual posts.
- Because there may be times where you loose the cell tower, this file creates a couple of files to log the data in case it can’t post. There is a topic file and a message file. Each time a post comes in for publishing it will first back it up to the files. Once it posts then it will clear the file. Does the same thing for power outages. If you have backed up messages that still need to post and you lose power, it will read those logs on boot and place them in a buffer to send. Once they all send it will clear the files.
within the main.py there are some print statements I had in there while troubleshooting this connection. You can comment them out if needed. I also connected my XCTU profile if you want to dump this to the XBee for settings establishment (profile -> apply profile). I’m on 11417 firmware so you may want to update to that if you want to use it. The only thing you should need to change would be your apn. I use twilio super sims and the apn is different.
Please let me know if you have any questions. I know this is a fair bit of info. more good info on the python aspect of XBee is here:
-
2021-08-03 at 11:09 AM #15769@vogelrnws Thank you for sending that. I’ll be sure to take a look at it if ThingSpeak doesn’t work.
@neilh20 I tried running the code with your ModularSensors library, and it sti@vogelrnws Thank you for sending that. I’ll be sure to take a look at it if ThingSpeak doesn’t work.
@neilh20 I tried running the code with your ModularSensors library, and it still didn’t work. The apn I’m using is hologram. If I am using a Hologram SIM card, is that correct, or should I change the source to something else? Also, what is Tera-term with Timestamp, and is that something I should consider using?And also, one more question. Is ThingSpeak with API different that ThingSpeak with MQTT? Right now, it looks like I’m using the MQTT connection.
-
2021-08-03 at 11:28 AM #15770@zeke-holloman
Everything I was talking about is MQTT with Thingspeak. I tested it on my Thingspeak connection I created. It was just allowing the micro-python code in the Xbee to handle all the ne
@zeke-hollomanEverything I was talking about is MQTT with Thingspeak. I tested it on my Thingspeak connection I created. It was just allowing the micro-python code in the Xbee to handle all the network and MQTT items vice using the AT commands from the MayFly to the Xbee as it looks like is happening in the enviroDIY libraries.
-
2021-08-03 at 11:30 AM #15771@zeke-holloman do you want to send me the thingspeak details on my email neilh20@wLLw.net and I’ll try them in my setting. I have@zeke-holloman do you want to send me the thingspeak details on my email neilh20@wLLw.net and I’ll try them in my setting. I have a historical thingspeak account which has exceeded its free quota, so I can’t create more devices.
The hologram account should work with the Xbee LTE ( I have an XB3-CA2-UT-001) if setup, and I started with hologram. If it works for getting time, should work for any other connection. If you post the trace can see if its getting good response .
@vogelrnws gosh sounds interesting, especially as you are logging the readings. I couldn’t see the python files – could you provide a link.
-
2021-08-03 at 11:41 AM #15772@neilh20
I’ll put the files up on a google drive and send a link for a folder to download here in a bit.
Yes, micro-python gives you lots of option. We use TLS1.2 encryption of our data, run a
@neilh20I’ll put the files up on a google drive and send a link for a folder to download here in a bit.
Yes, micro-python gives you lots of option. We use TLS1.2 encryption of our data, run a broker at one of our regional headquarters and also have a FTP-Secure server to remotely update both the XBee and the Teensy 3.5 processor I use as the datalogger. The Teensy runs a special bootloader that allows it to boot off the SD Card file. So I can pull the new file from the server with the XBee and then transfer via the serial port to the Teensy as it writes it to the card. Reboot and boom, new program is running.
-
2021-08-03 at 12:33 PM #15776@vogelrnws sounds great .. the Teensy 3.5 is a great board and amazing support. If you’ve figured out remote upgrade as well, very no@vogelrnws sounds great .. the Teensy 3.5 is a great board and amazing support. If you’ve figured out remote upgrade as well, very noce. I wonder if you would create a separate thread for it ~ perhaps “using the Teensy 3.5”. One of the biggest issues for me is always how to make a system, low power and solar powered.
-
2021-08-03 at 1:07 PM #15780
@neilh20 Ok, that sounds good. I just sent the details to your email. Let me know if it still works.
-
2021-08-03 at 1:08 PM #15781@neilh
Teensy 3.5 topic set up.
@Both
google drive link for the XBee files if either one of you want them in the future.
https://drive.google.com/drive/folders/1cr6PCJJRH8-qYIqOKSRQzYnqL8qg4nC5?us
@neilhTeensy 3.5 topic set up.
@Both
google drive link for the XBee files if either one of you want them in the future.
https://drive.google.com/drive/folders/1cr6PCJJRH8-qYIqOKSRQzYnqL8qg4nC5?usp=sharing
-
2021-08-05 at 9:41 AM #15801@vogelrnws I was spending some time looking at the cellular stuff and I decided I’d try to use the python example you used, but I can’t seem to access the files through the Google Drive. A@vogelrnws I was spending some time looking at the cellular stuff and I decided I’d try to use the python example you used, but I can’t seem to access the files through the Google Drive. Also, I don’t think I currently have a python editor. What is a good one that you would suggest? Thanks
-
2021-08-05 at 9:50 AM #15802
it seems I have to add your contact email to allow the share. I’ll see if I can post the zip file. It didn’t like the .py files before
-
2021-08-05 at 9:52 AM #15803
@vogelrnws Ok, thank you. If you need my email, zholloman54@gmail.com
-
2021-08-05 at 9:55 AM #15804
-
2021-08-05 at 10:06 AM #15805
Awesome, thank you. That worked
-
2021-08-05 at 10:49 AM #15806
@vogelrnws Hi there! So I uploaded the files onto the Xbee and used a python editor to change the main.py file. Is there any arduino code you would recommend using alongside this approach?
-
2021-08-05 at 11:23 AM #15807Any arduino code you would like to use. The key is to identify which serial port you have connected to the XBee. For instance if you have it connected to Serial1 then you would want to print out the sAny arduino code you would like to use. The key is to identify which serial port you have connected to the XBee. For instance if you have it connected to Serial1 then you would want to print out the statements you want sent to Thingspeak like so…
Topic posting to includes multiple fields
Serial1.println(#Pfield1=60&field2=45)
Topic posting to is field specific to field1:
Serial1.println(#P60)
The #P is the identifier the xbee python code is looking for to identify the data you want sent. the println() includes the \r\n characters that allow the python code to separate multiple post request coming in one after the other.
-
2021-08-05 at 11:56 AM #15808
@vogelrnws Ah, ok! Awesome, thank you. And is there a place to put the apn in the main code?
-
2021-08-05 at 12:07 PM #15809No, the apn would need to be loaded via XCTU or you can place it in the python code before the cellular connection action…
xbee.atcmd(“AN”, “hologram”) —&#
No, the apn would need to be loaded via XCTU or you can place it in the python code before the cellular connection action…xbee.atcmd(“AN”, “hologram”) —–or whatever your apn is
conn = network.Cellular()
-
2021-08-05 at 10:40 PM #15816@zeke-holloman I’ve been out the office. Just to summarize our offline investigation – I got a good POST using your thingsp@zeke-holloman I’ve been out the office. Just to summarize our offline investigation – I got a good POST using your thingspeak settings. It seem to me that its your modem data plan that isn’t connecting in some way. Maybe there is a better error indication from DigiXBeeCellularTransparent, but its an evolving area to make it easy.
-
2021-08-06 at 11:57 AM #15818
Ok, thanks Neil. I’ll look into it.
-
2021-09-14 at 9:56 AM #15899@neilh20. Hey Neil! Sorry it’s been a while, but just a quick update. I was able to get it to work. I ended up purchasing a new XBee cellular modem (same type) alongside the Digi IoT Development@neilh20. Hey Neil! Sorry it’s been a while, but just a quick update. I was able to get it to work. I ended up purchasing a new XBee cellular modem (same type) alongside the Digi IoT Development Kit. Works great now!
I’m thinking the issue was either that I had set up the modem with a board that wasn’t the dedicated XCTU board (I was using the Xbee Grove Development board). Either that, or I just messed up the set-up with the Xbee from the get-go.
Thanks for all your help!
-
2021-09-14 at 10:59 AM #15900@zeke-holloman great glad to hear it.
Which modem did you buy and where did you get it?. I thought they were in short supply, and I&
@zeke-holloman great glad to hear it.Which modem did you buy and where did you get it?. I thought they were in short supply, and I’ve been keeping my eye open for them.
I’ve been using the Digi LTE XB3-C-A2-UT-001, which needs the the latest XCTU modem board to upgrade as its is a complex process.
-
2021-09-14 at 11:17 AM #15901That is the version of XBee I use as well. They are very short supply. I only have some cause we bought a large supply of them bout a year ago. The dev board I use with mine to upgrade the cellularThat is the version of XBee I use as well. They are very short supply. I only have some cause we bought a large supply of them bout a year ago. The dev board I use with mine to upgrade the cellular firmware and normal XCTU operations is the XBIB-CU-TH. Digi-Key has 200+ of those.
-
-
AuthorPosts
- You must be logged in to reply to this topic.
