Welcome to EnviroDIY, a community for do-it-yourself environmental science and monitoring. EnviroDIY is part of WikiWatershed, a web toolkit designed to help citizens, conservation practitioners, municipal decision-makers, researchers, educators, and students advance knowledge and stewardship of fresh water. New to EnviroDIY? Start here

Manual upload of CSV files on MonitorMyWatershed

Home Forums Mayfly Data Logger Manual upload of CSV files on MonitorMyWatershed

Viewing 7 reply threads
  • Author
    Posts
    • #14019
      Robert S
      Participant

      I don’t know if this is related to the issue reported by @mbarney (monitormywatershed.org is not responding) or to the issue posted on GitHub by @dbressler (monitor my watershed won’t load, just spins #477) but manual uploading of CSV files is not working either.

      The page sits idle until the “Failed to upload data” message appears at the bottom of the page.  I have tried several time over the past couple of days, but with no change.

       

      Robert

    • #14023
      Matt Barney
      Participant

      Hi Robert,

      I just now tried and am also unable to upload a CSV. In the past, I had trouble due to formatting or due to files that were too long, but once I figured that out, I saved examples of files that I’ve successfully uploaded. However, those are now failing as well. Sample csv attached, FWIW.

      Matt

    • #14025
      Robert S
      Participant

      Thanks for the verification Matt.

      I was aware that large files can be a problem.  I am trying to upload 2 days worth of readings (about 290 lines) but see your file is even smaller.

       

      Robert

    • #14033
      Robert S
      Participant

      Well, I have pretty much answered my own question.

      After doing some searching on GitHub, I came across Issue #303 ( Allow POST of data, even when one variable UUID is wrong), which got me looking hard at the .csv data I was trying to upload. It really didn’t become clear to me until I grabbed some new data from the sensor last night.

      If you have four headers defined (time, water temp, voltage, board temp) and you drop the last one so there are only three data fields, the upload will fail. Just one of the pitfalls of touching the data too much.

      Funny in a way because I had changed the code in my logger (some time ago) to output the exact format I need to upload but haven’t been out to the sensor to upload the new code.

       

      Robert

    • #14034
      Sara Damiano
      Moderator

      The csv uploading tools are very sensitive to an irregularities in the files.  We would very much like to improve them, but right now that’s pending time and, mostly, budget.

    • #14035
      Robert S
      Participant

      I think that sensitivity is a good thing when collecting data and I want the system to tell me I am doing something wrong. If I could change anything, it might be to tell me what I am doing wrong.

      Either way, this was a case of “User Error” and Bad Practice Management.  I need to get my logger updated so that the .cvs file it is producing is in the EXACT format it needs to be for uploading. Then I don’t have to fiddle with copy&paste and run the risk of corrupting or altering the data inadvertently.

       

      Robert

    • #14043
      Sara Damiano
      Moderator

      It would be so much better if the website at least told you *why* and upload failed!  All you get is a “failed” which could mean anything – the format was bad, the file was too big, there was an internet blip, who knows.  The only way right now to even get an inking as to why it fails is to turn on “developer tools” (F12 in Firefox, I think also in Chrome) in your browser before posting the data and then searching through all the many tiny requests on the console for the right post request and hoping the error message is there and is interpret-able – and often it isn’t.

    • #14045
      Robert S
      Participant

      Hmm, turning on the console never occurred to me. I will have to remember that one.

      Thanks for your insight and knowledge Sara. I appreciate all of the useful conversation.

       

      Robert

Viewing 7 reply threads
  • You must be logged in to reply to this topic.