Library alternatives to achieve same goal as dataLogger in Modular Sensors

Tagged: googlesheet, private

This topic has 6 replies, 3 voices, and was last updated 2025-11-25 at 6:11 PM by neilh20.

Viewing 6 reply threads

Author

Posts

2025-11-21 at 10:30 AM #19435
cameron light
Participant
Hello. Our end goal is a 15-minute sampling cycle that sends data only via POST to a private Google Script webhook.
Here is where we stand. The modem now wakes up and transmits the sensor payload directly to the Google Script webhook.
I believe our issue pertains to the Sleep/Alarm System. The system struggles to enter and maintain the deepest sleep mode required for low power. Specifically, the RTC alarm and interrupt sequence consistently fails to cleanly disengage the wake line, resulting in a re-wake loop. I understand that the dataLogger in modular sensors handles all this. I don’t think the cpu is ever going into sleep and regardless of the time interval I give, the measurements are about 60-90 seconds apart.
And so, my question is, does anyone have any suggestions to completing this without the Modular Sensors library? I think we need an alternative because of conflicting libraries. Or is there a different approach? Has anyone done this before in the forum? I would appreciate any guidance on this.
Right now I’m using the following libraries: Timelib, DS3232RTC, TinyGSM
Thank you!
2025-11-24 at 5:55 PM #19440
neilh20
Participant
Hi @cameronlight, gosh fascinating to do some lateral thinking.

You would need to publish your whole program – preferably on a traceable git, like github.com.

The challenge in software is abstracting the complicated stuff (service layers) to make it look simple by encapsulating it into a simple interface.

Its always fascinating when someone knows enough to be able to use some new “libs” to do some magic.

From knowing modular sensors architecture – though I haven’t touched for a couple of years after I ran out of program space on the Mayfly – what you are looking to do is create a new publisher for the “POST to a private Google Script webhook.” That is it needs to be the service layer for POSTing to your chosen destination.

As you are identifying, there are a lot of infrastructure layers to keep the Mayfly on a low power even keel. As is often said, you learn more by your failures.

Of course when it all works its nice to describe it for others to discuss/try. 🙂
2025-11-25 at 11:33 AM #19441
cameron light
Participant
Hello @neilh20 thanks for the reply!

I’ve uploaded one older script to github.com at https://github.com/WEN-Research-Group because this script is getting the data to Google Sheets, and still using the Modular Sensors ecosystem, but the timing is off and it is never entering sleep. It’s just a single file in a repo.

Here’s the broader problem I am trying to solve: the data we are collecting must be stored and accessed through our server due to agreements with the owners of the land. Can you think of any other ways to achieve this?

Also, I’m now leaning towards sticking with the Modular Sensors ecosystem. So would creating a new publisher involve going in and rewriting the publisher .h and .cpp files?

Thank you
2025-11-25 at 11:34 AM #19442
cameron light
Participant
Apologies, this is a link for the repo https://github.com/WEN-Research-Group/Mayfly-to-Sheets
2025-11-25 at 3:56 PM #19443
Shannon Hicks
Moderator
The Sodaq_DS3231 library should work with the current Mayfly boards. Here’s an example from a few years ago that does basic microSD card logging of the onboard sensors and doesn’t rely on ModularSensors: https://github.com/EnviroDIY/EnviroDIY_Mayfly_Logger/blob/master/examples/mayfly1_TempHumidLight_simpleLogger/mayfly1_TempHumidLight_simpleLogger.ino
2025-11-25 at 4:06 PM #19445
neilh20
Participant
@cameronlight, very interesting needing to be clear about security of readings. Hard to do but a requirement for some property owners.

I was interested in doing something with sheets, however it was to learn about Oauth2 security as I remember. Obviously security access for google is core to the simplicity of their interfaces/APIs – but how does it work. Still evolving for me.

What I would do with this, is attempt to prototype it with CURL (or any other CLI) which can provide a log of all its internet activity. When there is a reliable script, running on the local PC that can POST to google sheets, with a traceable log, that takes into account security renewables and timeouts, then you have the algorithm that is needed.

Then translate that algorithm to C++ (publisher subsystem) running on the Mayfly.

I tried asking these questions to get some insights

gemini.google.com: Create a curl script with Oauth2 access to post values to google sheets
Answer Analysis : identifies an Oauth2 Access Token and a Refresh Token. Access tokens expire (typically in 1 hour). To automate this process without human intervention, you must first get a Refresh Token and then use it to request a new access token.
https://gemini.google.com/app/2c95df171b1a80b1 (it gave me this reference, may only work if logged in. Didn’t work for me when moving to another browser that I wasn’t logged in on)

2025-11-25 at 6:11 PM #19446

neilh20

Participant

@cameronlight, I had a suggestion to try replit.com – enabled an account and it allows 10apps to be built free.

I tried “Create a curl script with Oauth2 access to post 3 values to google sheets”

I was logged into my google account, for latter testing. When asked to allow replit permissions pretty have to allow it all it seems, particularly read/write spreadsheets.

This was the script it created. I ended up being able to add 3 values to a spreadsheet in my account.

#!/bin/bash

SPREADSHEET_ID="${1:-YOUR_SPREADSHEET_ID}"
RANGE="${2:-Sheet1!A1:C1}"
VALUE1="${3:-Hello}"
VALUE2="${4:-World}"
VALUE3="${5:-123}"

get_access_token() {
    local hostname="$REPLIT_CONNECTORS_HOSTNAME"
    local x_replit_token=""
    
    if [ -n "$REPL_IDENTITY" ]; then
        x_replit_token="repl $REPL_IDENTITY"
    elif [ -n "$WEB_REPL_RENEWAL" ]; then
        x_replit_token="depl $WEB_REPL_RENEWAL"
    else
        echo "Error: No authentication token found" >&2
        exit 1
    fi
    
    local response=$(curl -s "https://${hostname}/api/v2/connection?include_secrets=true&connector_names=google-sheet" \
        -H "Accept: application/json" \
        -H "X_REPLIT_TOKEN: ${x_replit_token}")
    
    local access_token=$(echo "$response" | grep -o '"access_token":"[^"]*"' | head -1 | cut -d'"' -f4)
    
    if [ -z "$access_token" ]; then
        echo "Error: Could not retrieve access token. Make sure Google Sheets is connected." >&2
        echo "Response: $response" >&2
        exit 1
    fi
    
    echo "$access_token"
}

post_values_to_sheet() {
    local spreadsheet_id="$1"
    local range="$2"
    local val1="$3"
    local val2="$4"
    local val3="$5"
    local access_token="$6"
    
    local json_body=$(cat < [range] [value1] [value2] [value3]"
    echo ""
    echo "Examples:"
    echo "  ./post_to_sheets.sh 1BxiMVs0XRA5nFMdKvBdBZjgmUUqptlbs74OgvE2upms"
    echo "  ./post_to_sheets.sh 1BxiMVs0XRA5nFMdKvBdBZjgmUUqptlbs74OgvE2upms Sheet1!A1:C1"
    echo "  ./post_to_sheets.sh 1BxiMVs0XRA5nFMdKvBdBZjgmUUqptlbs74OgvE2upms Sheet1!A1:C1 Apple Banana Cherry"
    echo ""
    echo "To find your Spreadsheet ID:"
    echo "  Open your Google Sheet and look at the URL:"
    echo "  https://docs.google.com/spreadsheets/d/SPREADSHEET_ID/edit"
    echo "  The SPREADSHEET_ID is the long string between /d/ and /edit"
    echo ""
    exit 0
fi

echo "Retrieving OAuth2 access token..."
ACCESS_TOKEN=$(get_access_token)

if [ $? -ne 0 ]; then
    echo "Failed to get access token"
    exit 1
fi

echo "Access token retrieved successfully."
echo ""

post_values_to_sheet "$SPREADSHEET_ID" "$RANGE" "$VALUE1" "$VALUE2" "$VALUE3" "$ACCESS_TOKEN"

#!/bin/bash

SPREADSHEET_ID="${1:-YOUR_SPREADSHEET_ID}"

RANGE="${2:-Sheet1!A1:C1}"

VALUE1="${3:-Hello}"

VALUE2="${4:-World}"

VALUE3="${5:-123}"

get_access_token() {

local hostname="$REPLIT_CONNECTORS_HOSTNAME"

local x_replit_token=""

if [ -n "$REPL_IDENTITY" ]; then

x_replit_token="repl $REPL_IDENTITY"

elif [ -n "$WEB_REPL_RENEWAL" ]; then

x_replit_token="depl $WEB_REPL_RENEWAL"

else

echo "Error: No authentication token found" >&2

exit 1

local response=$(curl -s "https://${hostname}/api/v2/connection?include_secrets=true&connector_names=google-sheet" \

-H "Accept: application/json" \

-H "X_REPLIT_TOKEN: ${x_replit_token}")

local access_token=$(echo "$response" | grep -o '"access_token":"[^"]*"' | head -1 | cut -d'"' -f4)

if [ -z "$access_token" ]; then

echo "Error: Could not retrieve access token. Make sure Google Sheets is connected." >&2

echo "Response: $response" >&2

exit 1

echo "$access_token"

}

post_values_to_sheet() {

local spreadsheet_id="$1"

local range="$2"

local val1="$3"

local val2="$4"

local val3="$5"

local access_token="$6"

local json_body=$(cat < [range] [value1] [value2] [value3]"

echo ""

echo "Examples:"

echo " ./post_to_sheets.sh 1BxiMVs0XRA5nFMdKvBdBZjgmUUqptlbs74OgvE2upms"

echo " ./post_to_sheets.sh 1BxiMVs0XRA5nFMdKvBdBZjgmUUqptlbs74OgvE2upms Sheet1!A1:C1"

echo " ./post_to_sheets.sh 1BxiMVs0XRA5nFMdKvBdBZjgmUUqptlbs74OgvE2upms Sheet1!A1:C1 Apple Banana Cherry"

echo ""

echo "To find your Spreadsheet ID:"

echo " Open your Google Sheet and look at the URL:"

echo " https://docs.google.com/spreadsheets/d/SPREADSHEET_ID/edit"

echo " The SPREADSHEET_ID is the long string between /d/ and /edit"

echo ""

exit 0

echo "Retrieving OAuth2 access token..."

ACCESS_TOKEN=$(get_access_token)

if [ $? -ne 0 ]; then

echo "Failed to get access token"

exit 1

echo "Access token retrieved successfully."

echo ""

post_values_to_sheet "$SPREADSHEET_ID" "$RANGE" "$VALUE1" "$VALUE2" "$VALUE3" "$ACCESS_TOKEN"

This is a script worked in the replit sandbox.

./post_to_sheets.sh <spreadsheet_id> [range] [value1] [value2] [value3]

or per the example

./post_to_sheets.sh 1BxiMVs0XRA5nFMdKvBdBZjgmUUqptlbs74OgvE2upms Sheet1!A1:C1 Apple Banana Cherry

There maybe some REPLIT secret sauce, as its in a bash file, it does identify program flow.

The next step would be to see if it can be made to work on local PC. !!

Author

Posts

Viewing 6 reply threads

You must be logged in to reply to this topic.