Monday, February 17, 2014

Build Your Own Davis Weather Station Console!!!

This Davis weather station hacking stuff is addictive like crack cocaine is addictive.  I always seem to be taking another hit from the pipe.  I've figured out how to get into the Davis console's serial port (after which the two of us had a complete and frank discussion COMPLETELY IN UPPERCASE).  I then built my own Data Logger so I wouldn't have to buy the ridiculously expensive Davis dongle.  Then, after I figured out the wireless transmissions from the outdoor sensor suite, I made my own ISS receiver with a Pretty Pink Pager.  But of course, I can quit any time I want.  Me and Amy Winehouse.
"They tried to make me go to rehab I said No, No, No"  -  Bad Decision

Famous dead drug-addled musicians aside (and how long a list is that?  Don't they know that only Keith Richards is Immortal?), the path I was to go down became clear after Davis pulled a ridiculously stupid stunt in the infamous Firmware 3.0 release they started shipping in new consoles a while back.  The new firmware read 64 bytes from a fab-written, read-only section on the data logger's memory chip.  It took this data, did some algorithmic juggling, and then did a check against another 64 byte write-once block on the data logger put there by Davis.  If the console didn't like what it saw, it declared that the logger was invalid and refused to function further.

This was an obvious and misguided attempt to shut down a few hackers from building and selling third-party data loggers for a fraction of the price Davis charged.  Problem #1 was that people with an older, authentic Davis logger discovered that their pricey little dongle stopped working after a firmware "upgrade".  Problem #2 was it also disabled the serial interface connection I had discovered.  And that pissed me off.

Unfortunately for Davis, the console does not run a 2 GHz Core i7 processor capable of state of the art encryption and decryption algorithms.  It has a little Atmel Atmega processor running at 1 MHz or something like that.  It's abilities are somewhat more... restricted.  Like something that could be cracked by a 256 byte lookup table and a trivial bit of code. Hats off to Watson on WXForum for figuring this out. His most excellent hack is posted below for posterity.  Break out your Arduino and start cutting and pasting if you want to calculate a valid security code for a DIY logger.

u8 const GreenDot_Table[256] =
{
0x00, 0x04, 0x08, 0x0C, 0x10, 0x14, 0x18, 0x1C, 0x21, 0x25, 0x29, 0x2D, 0x31, 0x35, 0x39, 0x3D,
0x46, 0x42, 0x4E, 0x4A, 0x56, 0x52, 0x5E, 0x5A, 0x67, 0x63, 0x6F, 0x6B, 0x77, 0x73, 0x7F, 0x7B, 
0x8C, 0x88, 0x84, 0x80, 0x9C, 0x98, 0x94, 0x90, 0xAD, 0xA9, 0xA5, 0xA1, 0xBD, 0xB9, 0xB5, 0xB1, 
0xCA, 0xCE, 0xC2, 0xC6, 0xDA, 0xDE, 0xD2, 0xD6, 0xEB, 0xEF, 0xE3, 0xE7, 0xFB, 0xFF, 0xF3, 0xF7,
0x18, 0x1C, 0x10, 0x14, 0x08, 0x0C, 0x00, 0x04, 0x39, 0x3D, 0x31, 0x35, 0x29, 0x2D, 0x21, 0x25,
0x5E, 0x5A, 0x56, 0x52, 0x4E, 0x4A, 0x46, 0x42, 0x7F, 0x7B, 0x77, 0x73, 0x6F, 0x6B, 0x67, 0x63,
0x94, 0x90, 0x9C, 0x98, 0x84, 0x80, 0x8C, 0x88, 0xB5, 0xB1, 0xBD, 0xB9, 0xA5, 0xA1, 0xAD, 0xA9,
0xD2, 0xD6, 0xDA, 0xDE, 0xC2, 0xC6, 0xCA, 0xCE, 0xF3, 0xF7, 0xFB, 0xFF, 0xE3, 0xE7, 0xEB, 0xEF,
0x31, 0x35, 0x39, 0x3D, 0x21, 0x25, 0x29, 0x2D, 0x10, 0x14, 0x18, 0x1C, 0x00, 0x04, 0x08, 0x0C,
0x77, 0x73, 0x7F, 0x7B, 0x67, 0x63, 0x6F, 0x6B, 0x56, 0x52, 0x5E, 0x5A, 0x46, 0x42, 0x4E, 0x4A,
0xBD, 0xB9, 0xB5, 0xB1, 0xAD, 0xA9, 0xA5, 0xA1, 0x9C, 0x98, 0x94, 0x90, 0x8C, 0x88, 0x84, 0x80,
0xFB, 0xFF, 0xF3, 0xF7, 0xEB, 0xEF, 0xE3, 0xE7, 0xDA, 0xDE, 0xD2, 0xD6, 0xCA, 0xCE, 0xC2, 0xC6,
0x29, 0x2D, 0x21, 0x25, 0x39, 0x3D, 0x31, 0x35, 0x08, 0x0C, 0x00, 0x04, 0x18, 0x1C, 0x10, 0x14,
0x6F, 0x6B, 0x67, 0x63, 0x7F, 0x7B, 0x77, 0x73, 0x4E, 0x4A, 0x46, 0x42, 0x5E, 0x5A, 0x56, 0x52,
0xA5, 0xA1, 0xAD, 0xA9, 0xB5, 0xB1, 0xBD, 0xB9, 0x84, 0x80, 0x8C, 0x88, 0x94, 0x90, 0x9C, 0x98,
0xE3, 0xE7, 0xEB, 0xEF, 0xF3, 0xF7, 0xFB, 0xFF, 0xC2, 0xC6, 0xCA, 0xCE, 0xD2, 0xD6, 0xDA, 0xDE
};

u8 const Adesto_Factory_Programmed[64] = 
{
  // put here AT45DB011D Security Register from byte 64 to 127
};

u8 OneTime_User_Programmable[64];

OneTime_User_Programmable[0] = 0x00; // or whatever you want
OneTime_User_Programmable[1] = 0x00; // or whatever you want
OneTime_User_Programmable[2] = 0x00; // or whatever you want

u8 n, value;

for(n=3;n<64 br="" n="">{
 value = (u8)(Adesto_Factory_Programmed[n] + n);
 OneTime_User_Programmable[n] = GreenDot_Table[value];
}

For more on this whole sad saga, you'll want to read this masterwork by torkelmj, especially if you enjoy reading stuff dripping with spite and vitriol.  Like I do.  Or skip that and just buy a security-coded third party logger from WXForum awesomedude Belfryboy.  Or order up his PCB on oshpark and build your own.  There are options now, and options are good.

Now one might think that everything is just peachy now that DIY loggers are possible once again.  Not me.  I was pretty grumpy that Davis had tried to screw their customers over even if they did release subsequent firmware updates that tried to undo some of the damage.  Once burned, twice shy, so they say.  I started thinking of a Plan B.  I want to build my own console.
Gratuitous Picture of My Dog
My IM-ME thing was a neat hack, but that is about as far as it goes.  IM-ME's aren't actually rolling off the production lines in millions of units per month these days, and their ability to hook up to other little do-dads would not be easily done.  Cheap transceiver boards based off of the popular CC1101 chip that work in the 915 MHz band are hard as hell to come by.  Ask me how I know.  The Seeedstudio RFBee comes tantalizingly close, but others who have tried to go this route run out of RAM and Flash on the limited processor found on this unit before getting too far.

I needed another way and that led me to the Moteino with its awesome little RFM69B transceiver module.  I got it picking up Davis ISS transmissions earlier this year.  Since then I have been coding away in my spare time and have been pulling together the various bits of hardware that I need to get something going.  Something like this.
Beautiful, Isn't It?

Some cool stuff has sprung up from this very breadboard.  My serial connection to the console grew up there on the left side.  The DIY logger design is in the middle.  The stuff on the right half starting from the left is the humidity sensor, the pressure sensor module, the Moteino itself, and finally my USB to Serial Adapter for the project I'm writing about today.

Now I know what you are saying to yourself: "The title of the blog is "Build Your Own Davis Weather Station Console!!!" but that doesn't look like a console at all!  Where is the keyboard?  Where is the display?" Well, I guess I'm not going to get anything past you today.  There isn't a keyboard or a display... unless you want to add one.  The hardware and software here is all open for you to add on what you'd like.  Me, I'm thinking of hooking this bad boy to a serial port on my Beaglebone running Openhab and having all my weather data available from anything in my house that can bring up a web browser.  Very doable, and no display or keyboard necessary for that kind of application.

But why call it a "Console"?  Because I couldn't think of a better name.  And I did indeed try.  It isn't a "Receiver" because it does more than receive given that it has its own sensors hanging off of it.  "Indoor Unit" sounds stupid.  "Thingamahooie" was tempting, I must admit, but didn't quite convey the application well enough.  And hey, the guy that builds the hardware, develops the code, and writes the blog gets to call what he comes up with whatever the hell he wants.  I'm going with "Console", and you'll just have to deal with it.

But I, as I so often do, digress.  Back to business.  On the hardware side of things, here is what I'm working with in more detail:
  • Moteino with an RFM69 module on it.  Pick the 915 MHz RFM69W (standard transmit power) or RFM69HW (high transmit power) at your option.  I have the serial version but it probably would have made sense to pick up the USB variant so I wouldn't need to tie up my USB-Serial converter all the time.  You probably want to add on the male headers option (why even think about it for the one dollar price?) and the flash memory option for four dollars.  I'm not using that yet, but it might come in handy for data logging at some point.  And if you re-dedicate the Moteino to something else later, this memory can be used for wireless remote programming.  Cool.
  • a DHT22 Temperature and Humidity Sensor for less than five bucks.
  • a BMP085 Temperature and Pressure Sensor for less than six bucks.  But it looks like that module is now much cheaper thanks to the compatible BMP180 sensors now out there for under three bucks.  The two are compatible, and I'd probably go with the newer BMP180 myself just because it is smaller.
  • a (yet to arrive and be connected) DS3231 Real Time Clock and Memory Module for less than three bucks (I felt bad for getting something at this price with free shipping so I bought two).  Note that this module also has a 24C32 EEPROM on it that is bound to come in handy.  Don't be dumb and order a DS1307 clock module instead: that one doesn't work well with the 3.3V Moteino and it is horribly inaccurate.  What good is a clock chip that can't keep time?
Arduino is all about little software libraries, and these are the ones I'm working with right now.
  • the LowPowerLab RFM69 library.  Felix has merged the changes I've needed so no need to use my fork of his code any longer.  I don't know if I'll be using this in the long term or come up with my own that is better suited for this application.  Another library that I'm including right now but not really using yet is his SPIFlash library.
  • the JeeLabs DHT22 code.  I have extracted what I need from the JeeLib code and included it into my own library.  JCW's stuff is a leaner and meaner version of some of the other ones floating around on the net.
  • the Adafruit BMP085 library.  I used this as is.  I did look at using the JeeLabs code but it was a little too tied up with JCW's PortsLib for me to bother untangling at the time.
  • the Arduino Serial Command library to parse command input.  This is a really great little library that lets you register callbacks for each separate command and also does command line argument parsing.  I needed to make a few tweaks as noted below, but nothing major.  My changes have been pushed but not merged at this time, so use my fork here.
  • my own DavisRFM69 library, of course.  I've got to be doing something to make all this stuff work together.
Enough chitchat.  Let's cut to the chase, shall we?  Take a look at this thing actually doing something.
Success - I Never Tire of It
This is SandaySoft's Cumulus software talking to my homebrew weather station console.  The outdoor readings are those from my ISS in the front yard.  The indoor readings are from the sensors I have hanging off the Moteino.  On the left you can see the Serial Port Monitor Software recording the RS-232 messages going back and forth.  Cumulus has no idea that it isn't talking to a real Davis console.

Thanks to the Davis Serial Protocol Manual, getting this working wasn't too terribly difficult.   I knew Cumulus used the Davis SDK to talk to the console so the Cumulus author might not be aware of every single command being sent to the station and back.  I also knew that the LOOP command was used to get the data out of the station so I implemented that and some other useful commands on the Moteino.  Then I hooked my real console up and watched some packets go by and found a few more I needed to implement (namely BARDATA, HILOWS, and RXCHECK).  I had a few hiccups of course.  For example, Davis says that a command should always be terminated with '\n' (0x0a) like this...
 except when they don't, like this...
And that is the fun with emulation: you have to emulate not only the functionality, but the bugs as well.  This one caused me to tweak the serial port library to accept multiple command terminators.  Another oddity is that the console itself goes to sleep all the time and is woken by simply sending a lone '\n' that the console is supposed to acknowledge.  The serial library I'm using hadn't actually accounted for that situation.  Now it does.

Want to play along?  Here is what you need to do.   From within your Arduino's libraries directory...
  • Get my version of the SerialCommand library that supports empty commands and a second line terminator.  And of course, you'll want to get a copy of my DavisRFM69 library.  The pared down DHT22 library is tucked away inside the latter.
git clone https://github.com/dekay/Arduino-SerialCommand.git SerialCommand
git clone https://github.com/dekay/DavisRFM69.git
git clone https://github.com/adafruit/Adafruit-BMP085-Library.git Adafruit_BMP085
git clone https://github.com/LowPowerLab/RFM69.git
git clone https://github.com/LowPowerLab/SPIFlash.git
Hardware hookup is like this.

DHT22 Pin 1 -> Moteino 3.3V Out
DHT22 Pin 2 -> Moteino D4 with 10K Pullup resistor to Moteino 3.3V Out
DHT22 Pin 3 -> Not connected
DHT22 Pin 4 -> Moteino GND

BMP085 Module Pin VCC -> Moteino 3.3V Out
BMP085 Module Pin SDA -> Moteino SDA
BMP085 Module Pin SCL -> Moteino SCL
BMP085 Module Pin GND -> Moteino GND
Once you've got all of this hooked up, you'll find a "VP2.ino" sketch in the DavisRFM69 library's Examples directory.  Open that up in the Arduino IDE and select Control-U to compile and download it to the Moteino (be sure to tell the Arduino IDE that you are using an Arduino Uno).  Open up the serial monitor, set the baud rate to 19200 and "Newline" as the line ending.  You should see... absolutely nothing.  Nothing that is, until you type in some (uppercase) commands like "LOOP 1", "TEST", or "STRMON".  The Moteino will respond exactly the same way as a real console would.  Hook it up to Cumulus (with the "Use Data Logger" option unchecked) and revel in a job well done.

I'm off to a decent start here but there is still lots to do.  Here are some things on my list.

  • integrate the real time clock functionality
  • put in the frequencies for consoles in Europe and other regions
  • add some more useful commands like PUTRAIN so you can tell this thing how much rain you've had this year that it can keep a running count of afterward
  • add in some kind of EEPROM functionality so it retains settings like total rainfall between power downs.  Conveniently, there is an EEPROM on my yet to arrive three dollar clock module just begging to be used.
  • make the reception more resilient.  There are some odd things going on here: I get a spurious interrupt on startup (setup issue?) and when I move the board around (need a pulldown on the interrupt line?)  Reception is also poor during the day.  Might be that I just need to adjust the clear channel RSSI value a bit to accommodate more background noise.
  • integrate the real time clock functionality (daily / weekly / monthly / year highs and lows)
  • etc
Anyone that has done much coding on the Arduino gets to know memory limits.  The Moteino has double the RAM and EEPROM than the standard Arduino UNO, so I've got a whopping 2K of RAM and 32K of Flash to play with.  So far I have used roughly half of each, so I've got some headroom.  I haven't done much to try and streamline any of the code besides a few const arrays here and strings stuffed in the F() macro there, so I am sure there is room for improvement if things get really tight.

With this project, I think I've hit the goals I set out to achieve: come up with something functional, inexpensive, and with a low barrier to entry so that other people can dig in.  Have a new feature you'd like to see?  Send me a pull request on Github.  Want to develop a plug in circuit board (kind of like this) with all the dodads attached to lower the barrier to entry even further while making a few bucks on the side?  Be my guest.

I'll wrap this up by saying that the Davis guys make good stuff.  The ISS is very well put together and I love that the thing is solar powered with battery backup.  I've had mine for years and it has been operating trouble free on Battery #1.  The display on the indoor console is nice, big, and easy to read and it also just sips on battery.  I'm guessing that 99.9% of Davis' customers are going to continue buying standard Davis stuff off the shelf despite whatever I come up with here.  Anything I play around with  is ony going to be picked up by that other 0.1% of enthusiasts.  And it is those enthusiasts that are not only going to continue buying Davis products but will recommend them to their friends as long as they are kept happy.  If Davis pulls another stupid Firmware 3.0 thing, they'll do more damage to themselves than I ever did.  I hope they continue documenting their communication protocol and maintaining accessible expansion ports in their current and future products.  That would be a win-win for everybody.

With that in mind, it seems as good a time as any to unveil the new profile picture.
Be Nice


119 comments:

  1. Very cool. I will try to usw your code for a European Vue ISS.

    ReplyDelete
    Replies
    1. It'S Science, But It Works Like Magic.: Build Your Own Davis Weather Station Console!!! >>>>> Download Now

      >>>>> Download Full

      It'S Science, But It Works Like Magic.: Build Your Own Davis Weather Station Console!!! >>>>> Download LINK

      >>>>> Download Now

      It'S Science, But It Works Like Magic.: Build Your Own Davis Weather Station Console!!! >>>>> Download Full

      >>>>> Download LINK pH

      Delete
  2. Hi DeKay,
    I'm in EU testing with an Arduino Uno connected to an RFM69W(868Mhz) trying to receive a Vue ISS. I changed to the DAVIS_FREQS_EU but I'm currently not receiving anything, after typing LOOP it returns " LOO" and "±o". This should work with an Arduino Uno too, using voltage dividers to get 3.3V as one would connect an RFM12? The IRQ is connected to D2.
    Thanks

    ReplyDelete
    Replies
    1. Never mind... I seem to have modified the wrong DavisRFM69.h file, it works now :-)
      Thanks

      Delete
    2. Wow. I now have two users! That is two more than I ever thought I would have!

      Delete
  3. The Australian one is a modified U.S. device. Here in Oz the lower part of the 900Mhz band is used by mobile phone networks. On this forum http://forums.whirlpool.net.au/archive/1859768 about half way down a poster lists what is on the compliance sticker.
    915-928 MHz
    AUSTRALIAN MODEL
    ID Codes Available 8
    Output Power: Less than 8mW

    would this mean I could just use the U.S. setting in DavisRFM69.h

    ReplyDelete
    Replies
    1. Dunno. Maybe. Hopefully. Only one way to find out, and that would be to try it with the frequencies outside this range pruned out.

      Delete
  4. Great work! Valuable source of information for my future weaterstation project. I've added link to your site from my own blog (http://weatherhelge.wordpress.com)

    ReplyDelete
  5. I've been following your journey for months hoping to make use of the info gleaned to do some home monitoring, including weather of my home in MT. I've also been thinking about reading ubiquitous temp sensors from indoor/outdoor thermometers... That lead to wanting to snoop my power meter for electricity consumption (916MHz freq hopped MA). Your find of the Moteino looks to be capable of solving all these dreams!

    So my question to you is to ask for advice on snooping out the frequency hopping sequence. It seems you had the advantage of peeking at the console talking to the RF module changing channels. Do you think it will be possible to scan the channels to find the channel off the air?

    Joe Sestrich

    ReplyDelete
    Replies
    1. Possible, but certainly a challenge. First thing you'd want to do is see if you can decode the data at a single frequency (a lot harder than it sounds) and then see if you can figure out the hopping sequence. It may be that the sequence is more random than the fixed sequence used by Davis. If I were you, I would be looking at a cheap little USB stick to do software defined radio (eg http://www.nooelec.com/store/sdr/sdr-receivers.html) and something like SDR Sharp to give you a better idea of what is going on in the frequency domain. And I'd be trying to find out if anyone had done any prior research. Google is your friend here. Good luck.

      Delete
  6. I have no words to say how grateful I am for your work. During the last year I have developed a complete weather station based onArduinos, and now thanks to what I could learn by studying your code, I got the Cumulus recognizes my hardware as a VP2. Thank you so much for publishing it ... I will follow this subject closely.
    Werk_AG

    ReplyDelete
    Replies
    1. Nice! Have you written this up somewhere?

      Delete
  7. By now just design specifications (http://www.meteocercal.info/wx_weatherduino.php), the code is still not quite ready. Do you mind if I use parts of your code that emulate the Davis?

    Werk_AG

    ReplyDelete
    Replies
    1. I'm good with that as long as your source code is made available.

      Delete
    2. Thanks. I always publish the source code. The WeatherDuino Pro2 will be no exception.

      Werk_AG

      Delete
    3. The source code for the WeatherDuino Pro2 project is now published. The WeatherDuino Pro2 project allows you to build a full weather station with davis vp2 emulation.
      More info and source code here: http://www.meteocercal.info/forum/Forum-WeatherDuino-Pro2

      Thanks DeKay for your work.
      Werk_AG

      Delete
    4. That is pretty damn cool.

      Delete
    5. @Werk_AG "I always publish the source code. The WeatherDuino Pro2 will be no exception."
      I don't want to be rude, but have you really made your code publicly available? When I click at the link you are pointing (http://www.meteocercal.info/forum/Forum-WeatherDuino-Pro2) I get "This is a restricted access sub-forum". Do I have to pay for your pcb's, in order to gain access at your code?

      @Dekay Congratulations for your work, and forgive me for being off topic

      Delete
  8. Hi DeKay

    In VP2.ino:

    void cmdSettime() {
    byte davisDateTime[8];
    printAck();
    // delay(2000); Why were these delays here if read() is blocking? Kobuki code bug?
    // Read six bytes for time and another two bytes for CRC
    for (byte i = 0; i < 8; i++) {
    davisDateTime[i] = Serial.read();
    // delay(200);
    }

    In my case, if I don't uncomment the delay lines, my arduino rtc does not sync when Cumulus starts.

    Werk_AG

    ReplyDelete
    Replies
    1. Thanks for the heads up. I'll look into this and uncomment it in my master.

      Delete
  9. I'm confused about which RFM69 library to use, LowPowerLab's or yours. In the above blog post it says LowPowerLabs has incorporated your changes so you don't need yours. On your RFM69 github readme it says you don't need LowPowerLab library. So which should I use? My project is to get the wireless data from my Davis outdoor weather station and send it to my Weather Underground personal weather station account (using an Ethernet shield with the Moteino). I won't be using any of the indoor sensors like you have in your project.
    Great job on figuring all this out. It's quite an accomplishment.

    ReplyDelete
    Replies
    1. "I don't know if I'll be using this in the long term or come up with my own that is better suited for this application." Turned out to be the latter. See also.

      https://github.com/dekay/DavisRFM69/commit/400f5e12c97a5379ba534626d83c9e4aaad10161

      Good question! And thanks for the compliment.

      Delete
  10. The Moteino I ordered doesn't have the flash chip. So should I remove the flash code from VP2.ino? Or is there some other flash on the Moteino that this code is communicating with?

    ReplyDelete
    Replies
    1. The code uses the Moteino flash to store archive records and the EEPROM on the microprocessor to store configuration data.

      Delete
  11. In my project, I'm sending weather station data to Weather Underground; I won't be communicating with a console. I'm trying to understand the VP2.ino sketch. Is all the serial command stuff (sCmd) for communicating with the console? If so, I can remove it, right?

    ReplyDelete
  12. I'm up at my ski house this weekend and I've got the Moteino receiving data from my Davis weather station and displaying on the Arduino serial console. It seems to be working so far. Unfortunately it's dead calm, so I can't see any wind data. I had a couple questions. My battery status is 0, I assume that means the battery is okay (the battery is only a week old), but I wanted to double check. What's the battery status for a low battery, 1? Also, how do I get the rain data? I don't see anything in the example program for rain. Once I get this finished, I'll put on on github for others to use.

    ReplyDelete
    Replies
    1. For battery status: https://github.com/dekay/im-me/blob/master/pocketwx/src/protocol.txt#L51

      For rain: http://www.wxforum.net/index.php?topic=10739.msg223891#msg223891

      Delete
    2. Thanks for the links. I've got the Moteino working with a WIZ811MJ network module. Next steps are to grab the time from NTP server instead of a RTC chip and figure out what to upload to weather underground. Rain seems to be the most confusing.

      Delete
    3. BTW - I'm Scott216 in the wxforum

      Delete
  13. DeKay,
    I'm using your DavisRFM69.h library on the Moteino and it's working well except my sketch keeps locking up. It seems to happen when I'm making the Ethernet connection to weather underground. Does your library have any interrupts or anything that might interfere with the Ethernet connection? If so, do yo have any suggestions on how to fix?

    ReplyDelete
    Replies
    1. I should add that it everything works fine for 5 to 10 minutes. I did come across this post about an RFM12B messing up the Ethernet connection. Maybe I have a similar problem.
      I'm using a modified Ethernet library that a guy named SurferTim (on Arduino forum) created. He made a small change that let's the sketch choose a different SS pin for the Ethernet hardware. See bit.ly/1nvz2ye

      Delete
  14. I think I've got it working. I modified w5100 library as so setSS() disables interrupts with cli() and resetSS() enables them again with sei(). I've had the program running for about 30 minutes now and it hasn't locked up. Previously it would go about 5 minutes, 10 at the most. I'm not sure if there are unattended consequences with this mod, I hope not.

    ReplyDelete
    Replies
    1. Only one way to find out! It is very possible that there is some kind of interrupt contention going on that you have now worked around.

      Delete
  15. I'm trying to figure out the station ID. Your code says "The low order three bits are the station ID", I think you're talking about radio.DATA[0]. My station ID is 1, but the low order bits of DATA[0] are always zeros. Here's a few of them
    01100000
    10000000
    11100000
    01010000
    01000000
    01010000

    ReplyDelete
    Replies
    1. Davis' ID codes of #1 to #8 map to zero through seven in DATA[0].

      Delete
  16. Thanks. So transmitter ID 1 is 0 in the data. Did you ever determine if the transmitter ID changes the frequency hopping? Will your library work just as well with any ID?

    ReplyDelete
  17. It turns out the the frequency hop sequence is the same regardless of the ID. The difference is the timing between hops, which increments by 62.5 ms for every increment of the transmitter ID. Off the top of my head, I think the current code would work but I've never tested it, and I haven't looked at this stuff in a while. I get busy with yardwork during the summer and play with it again in the winter.

    ReplyDelete
  18. I finished up my project. I'v got the Moteino receiving the wireless data then it uploads it to Weather Underground with an Ethernet module. The project doesn't need the Davis console or PC. I've got everything on Github

    ReplyDelete
    Replies
    1. Github link isn't working in above post. Here it is:
      https://github.com/Scott216/Weather_Station_Data

      Delete
    2. Scott will it work with Vantage Vue or it is Vantage PRO intependant?
      Thanks!!!

      Delete
    3. Nikolaos,
      I don't know if this works with the ISS that comes with the Vantage Vue. It depends on if Davis transmits the data the same way is the Vantage Pro ISS.

      Delete
  19. Hi DeKay.
    How hard will this be for a guy who never worked with aurdino before to accomplish?
    Today i collect some temperatures with 1-wire sensors and graphs it using OWFS and RRDtool. My first plan was to connect my Vantage Vue console to the same ubuntu server and store the data from it in rrd databases. But getting the data directly from the Vantage Vue ISS sounds even better. But how steep is the learning curve doing it wireless compared to the wired solution?
    Thanks.

    ReplyDelete
    Replies
    1. Did you look at the Github repo I linked to above?
      http://git.io/Ds7oxw
      It's very easy to get the wireless data into the Arduino (thanks to DeKay's library). If you want to send it to weather underground like I did, then all the work is done. If you want to do something else with it like send it to your Ubuntu server, well I don't know how hard that is to do with an Arduino. If you want to push the data to Ubunto via HTTP GET, then it's probably pretty easy because you'd be getting the data the same way Weather Underground gets it. You'll need to know how to solder so you can connect the Moteino, Ethernet module and barometric pressure sensor together.

      Delete
  20. In the Davis Application note 28 (http://bit.ly/1qfT0vI) pg 12 it says the rain rate is calculated by the ISS and transmitted to the console. I'm already getting the rain counter, but I'd like the get the ISS's rain rate too now that I know about it. Do you know how to decode rain rate from the wireless ISS data?

    ReplyDelete
  21. For my project where the Moteino is uploading data to Weather Underground, I noticed that sometimes after the upload to WU, the radio takes a while to start getting the ISS data again. I know when I boot up the sketch it takes a while for the radio to first get the data. Is this because of the frequency hopping and it takes a while for the Moteino to "sync" up? If so, do you think it's likely that sending data to WU sometimes causes the radio to re-sync the frequency hopping?
    To get this project to work I had to modify the Ethernet and w5100 libraries a bit. I added the ability to choose the slave select pin and I temporarily disabled interrupts. This mod is causing some small problems, mainly I can't set the IP statically, only dynamically - which uses a lot more memory. So I'm thinking of using a separate Arduino to upload data to the internet. If I do this, I'll need to connect the Moteino to my Arduino to send it the ISS data. I'm considering two methods to do this. First is using I2C, second is software serial. I'm worried that interrupts in either of these methods might mess up the radio. I did find a non-blocking version of software serial library - AltSoftSerial that might be a better choice then the standard Arduino one. I don't know much about blocking libraries and interrupts. I'm hoping that with this change I could go back to the standard Ethernet library and I'd get rid of the radio syncing delay after uploading to WU. Do you have any advice for the best way to send the data form Moteino to an Arduino? Any thoughts on I2C, vs Software Serial vs AltSoftSerial?

    ReplyDelete
    Replies
    1. "Is this because of the frequency hopping and it takes a while for the Moteino to "sync" up?"

      Exactly. It is luck of the draw as to how long it takes, because it all depends on where the ISS is in the hop sequence when the Moteino starts listening. A real console has the same issue.

      "If so, do you think it's likely that sending data to WU sometimes causes the radio to re-sync the frequency hopping?"

      If you are sending data to WU and it takes longer than the time between packets, you could definitely lose an interrupt and get out of step. You'll need to restructure your code to prevent this.

      " This mod is causing some small problems, mainly I can't set the IP statically, only dynamically..."

      I see from the link you posted that you've now fixed this...

      "Do you have any advice for the best way to send the data form Moteino to an Arduino? Any thoughts on I2C, vs Software Serial vs AltSoftSerial?"

      Never communicated between Arduinos before (I find working with one is frustrating enough). Another option you haven't considered is a Moteino Mega with its second hardware serial port and more memory.

      Delete
    2. I noticed my Moteino has to resync the frequency hopping at least once every 3 minutes. I assumed it was because I was uploading data to Weather Underground. But as a test I commented out the Weather Underground upload it it didn't help the resyncing issue. Any thoughts? Does your setup behave similarly?

      Delete
    3. You might have one frequency out of the 51 that is out of line with mine. I figured the frequencies out from sniffing the SPI bus and not from figuring out the nominal values stored in the code. If it is always the same frequency (and they repeat every 2.5 * 51 seconds, and that is not far off three minutes), then you might need to tweak one a bit.

      Delete
  22. I didn't know their was a Moteino mega, Since I got the static IP resolved, that freed up a lot of memory. I'll try changing my code so it only uploads to WU immediately after I receive an ISS packet. Packets come in about every 10 seconds. I haven't timed how long it usually takes to upload to WU. Maybe that will help.

    ReplyDelete
    Replies
    1. "Packets come in about every 10 seconds."

      Where did you get this idea??? They come in 4x faster than that.

      Delete
    2. You're right. I have been focused on the rain data lately, and I get that every 10 or 11 seconds.

      Delete
    3. I've got a MoteinoMega, but I can't get it to receive any data from the ISS. I'm wondering if your RMF69 library is using the wrong SPI CS pin. On the MoteinoMega it's D2? I looked at your library and you define it like this:
      #define SPI_CS SS
      But I can't figure out where SS is defined. I'd appreciate any suggestions.
      Thanks.

      Delete
    4. SS is defined in the Arduino file "pins_arduino.h", not in my stuff. Make sure you have the right board specified in the Arduino IDE for it to pick up the right pin.

      e.g.

      [root@laptop share]# grep -r "static const uint8_t SS" *
      arduino/hardware/arduino/variants/mega/pins_arduino.h:static const uint8_t SS = 53;
      arduino/hardware/arduino/variants/leonardo/pins_arduino.h:static const uint8_t SS = 17;
      arduino/hardware/arduino/variants/robot_motor/pins_arduino.h:static const uint8_t SS = 17;
      arduino/hardware/arduino/variants/standard/pins_arduino.h:static const uint8_t SS = 10;
      arduino/hardware/arduino/variants/robot_control/pins_arduino.h:static const uint8_t SS = 17;

      Delete
    5. Looking into the SS pin, everything seems to be fine with the MoteinoMega, but I can't get it working with the Davis. I posted info on this in the lowpowerlab forum

      Delete
  23. I copied the RFM69registers.h file from the RFM69 library which seemed to be needed but there were still 4 #defines missing and giving "not declared" errors on compile. The missing items are:
    RF_FDEVMSB_4800 (But RF_FDEVMSB_5000 is in the RFM69registers.h file.)
    RF_FDEVLSB_4800 (RF_FDEVLSB_5000 is there too.)
    RF_AFCLOWBETA_OFF
    REG_TESTAFC
    I couldn't find these defined anywhere in any of the libraries mentioned in the readme or blog. Not sure how to fix this. :-(

    ReplyDelete
    Replies
    1. Oops. Yeah. You need to git clone LowPowerLabs RFM69 library into your libraries folder, not just copy the one file over into this directory.

      Or changing the include from

      #include <../RFM69/RFM69registers.h>

      to

      #include

      should work for the one file you copied over. I think.

      Delete
    2. Yeah, I did that and those four still seem to be missing. I still get four "was not declared in this scope" errors in 'void DavisRFM69::initialize()' on compile.

      Delete
    3. I see now that LowPowerLabs modified RM69registers.h which caused the problems. I added those lines (taken from the changes history) back into RM69registers.h and it now compiles fine.

      Thanks for your excellent work on this project. 8-)

      Delete
    4. Please could you help me?
      I have this error.

      error: prototype for 'void DavisRFM69::initialize()' does not match any in class 'DavisRFM69'
      void DavisRFM69::initialize()

      Delete
    5. Do you have my stuff in your Arduino libraries folder?

      Delete
    6. Yes.
      I download all your files from Github and extract in libraries folder.

      Delete
    7. could you walk us though on how you resolved this?. I ahve the exact same error

      Delete
    8. This comment has been removed by the author.

      Delete
    9. I use an older RFM69registers.h file with DeKay's library. You can find it here: http://git.io/xAcp

      Delete
  24. 50 is rain rate:
    http://www.wxforum.net/index.php?topic=18489.msg227753#msg227753

    There is a message 8 as well: http://www.wxforum.net/index.php?topic=18718.msg180578#msg180578

    40 and 60 are UV and Solar
    http://www.wxforum.net/index.php?topic=18489.0

    A couple of the others are probably soil moisture and leaf wetness that the Vue doesn't measure but the VP2 does.

    ReplyDelete
  25. > 50 is rain rate
    OK

    > 40 and 60 are UV and Solar
    I don't have messages 40 or 60
    I have 20, 30, 70, 90

    ReplyDelete
  26. I've tried to decode the rain rate packet but haven't really figured it out yet. It seems sometimes this formula works:
    byte3 + ((byte4 >> 4) - 4) * 255
    but it doesn't work for everything.
    Next time I have access to the weather station (it's at my vacation house) I'm gonna connect the rain collector to a relay that an Arduino controls and have a bunch of precisely timed bucket tips at different intervals. At least then I'll know exactly what the time is between bucket tips.

    ReplyDelete
  27. My brother has a wired Davis VP2; can the work you've done be used to extract data from the Davis console so he could then upload it to Weather Underground? Since my project focused 100% on the wireless data from ISS, I haven't looked at the wired portion at all.

    ReplyDelete
  28. I believe that STRMON prints the data from a wired station in the same format as the packets received wirelessly from your ISS. Tell him to give it a shot and see.

    ReplyDelete
  29. DeKay, thanks so much for your work/obsession. Forgive my ignorance if you've already done this, but have you ever looked into building a data receiver, say for the Vantage Pro 2, using Software Defined Radio (SDR)? A while back, I assembled a simple SDR setup using GNU Radio and $20 usb dongle on my linux laptop, and was able to construct a basic, graphical receiver for a couple of different kinds of RF traffic, for example FM radio and ADS-B traffic. I followed your posts and tried to view my Davis frequency-hopping data, but ran into a massive ignorance wall because I have zero training in RF / DSP engineering. Have you tried this, or have any interest? I'm decent at linux and coding, but can't offer much else tech-wise except volunteering to be a guinea pig on a linux box to help figure this out. And of course my kindred outrage and annoyance at Davis's evil regarding this device, which I had the misfortune of paying them for.

    ReplyDelete
    Replies
    1. I'm familiar with SDR and have my own $20 dongle that I've never yet played with. Too many fun projects to dig into and not enough time for one. For other, it seems kind of like overkill to use a PC running GNU Radio and an SDR dongle for something I can do with a lowly Moteino and likely with better reliability. That isn't to say it wouldn't be an interesting and fun project. I basically started out in this stuff from zero. I'm sure you could do the same :-)

      Delete
    2. Overkill.... Since when has that been a bar?? :-) It's not because I thought this was an 'efficient' or reliable way to go at all, especially considering one of the original annoyances with this product was that it required a second running computer to simply log data, which seemed the height of ridiculousness; or have to use their stupid web service. I was more throwing this out there because I need help in learning. I'm struggling to understand how at lot of this basic DSP works, and I thought if I could get this data into software (which is a tool I can use) at least then I could better see and play with it and reproduce what you've accomplished/learned in hardware (which I have almost *no* competency in). Thanks for your confidence in me that I could learn the hardware starting from scratch like you did, but it's more being faced with a choice between spending 2-3 years trying to steal occasional moments/weekends/late nights from my family getting up to speed versus maybe 2 weeks or months if I could get the raw signal into a computer to study and learn about it. Of course when I did try it I also pretty quickly ran into darkness there... hence my question to see if there was anyone out there who had tried it and had some experience to get me over the blocks. Afterall it was the only way I learned how to use computers in the first place, too. :-)

      Delete
    3. I would suggest the GNU Radio Companion, which interfaces to the SDR and provides a GUI from which you can use a building-block based approach to a receiver. That and Michael Ossman's tutorial series. You'll be pulling in data and analyzing it before you know it. Give it a shot. You could do it.
      http://greatscottgadgets.com/sdr/

      Delete
  30. After lots of testing I’m now quite sure to have figured out the meaning of the packets transmitted from the Davis Vue ISS.
    You'll find the results on my Homepage: http://www.carluccio.de/davis-vue-hacking-part-2/

    ReplyDelete
  31. I'm trying to compile VP2.ino, but I get the following

    VP2.ino: In function 'void setup()':
    VP2:113: error: 'class SerialCommand' has no member named 'setNullHandler'

    I'm not sure how to fix this.

    ReplyDelete
    Replies
    1. I just re-downloaded from your "the SerialCommand Library" link above and it compiled fine. Somehow I downloaded an older version the first time.

      Thank you very much for sharing all the hard work you put into figuring all of this out !

      Delete
    2. Glad you got it working. Sorry I didn't respond right away - I don't check this blog for comments as regularly as I should.

      Delete
    3. I'm from a much older programming era, so OO languages like C/C++ are very foreign to me. I've learned a lot from pouring over the sketches and libraries that you've shared - thank you. I'm really looking forward to any future work you do on this project.

      By the way, the picture above, showing the hardware hook up, that's stored on flickr shows that it's no longer available.

      Delete
  32. I created a simple version of your program which just gets the Davis RSS data and sends it to the serial monitor and also sends the time since the last packet was received. Normally there's about 2.7 seconds between data packets, but every 25th packet I get a CRC error and it takes about 70 seconds to recover from that. It's pretty consistent. Here's a link to the chart showing the time between packets
    http://bit.ly/1urXXY5

    Any ideas why these CRC errors happen so consistently. Do you see this on your setup? I'd really like to get rid of the CRC errors since every about every minute the program gets one and takes over a minute to recover, so I'm only getting about half the packets the ISS is sending.

    Here's a link to the program I used to run this test:
    https://gist.github.com/Scott216/bc0a77b91b5a8ddc6f61

    BTW - How do you embed pictures and URL links into these comments?


    ReplyDelete
  33. Start by asking yourself what you changed. Like this...

    const byte TRANSMITTER_STATION_ID = 3; // ISS station ID to be monitored. Default station ID is normally 1

    And then ask yourself what impact that might have on something else, like this...
    const uint16_t PACKET_INTERVAL = 2555;

    You are using an ID with a larger interval between packets but you didn't tell the rest of your code that. Start there.

    As for embedding URL links into comments, try this link
    http://lmgtfy.com/?q=url+in+blogger+comments

    ReplyDelete
  34. Thanks Dekay. I don't really know anything about frequency hopping or how the station IDs change things, but I found
    this forum post which talks a bit about it. It seems like I should add 62.5mS to PACKET_INTERVAL for each incremental station ID. I'll give that a try and see what happens.

    I did Google around about comments (I think I googled blogspot not blogger) and I tried a the <a href tag and it didn't work, but it worked this time, so I must have made a typo the first time and assumed I was doing it wrong.

    ReplyDelete
  35. Hello,

    Is possible find a list of all librarys needed for this project?
    Every time I install a library I need a other new.
    Now I need "variant.h"

    ReplyDelete
    Replies
    1. I don't have a variant.h anywhere on my laptop. Whatever is giving you this error, it isn't my stuff.

      Delete
    2. Not anywhere in my home directory on my laptop, that is. None of this stuff below is related to my code or Arduino stuff in general.

      [root@laptop dk]# find / -name variant.h|grep variant.h
      /usr/include/glibmm-2.4/glibmm/variant.h
      /usr/include/nepomuk2/variant.h
      /usr/include/strigi/variant.h
      [root@laptop dk]#

      Delete
    3. Finally works!
      I created a new Library folder and install all inside.
      Only needed little changes in some library.
      Good job!

      Delete
  36. Re:"Hardware hookup is like this." image is no longer available. I assume this is the schematic. Could you please repost. Nice job!

    ReplyDelete
    Replies
    1. Crap. I don't think that picture was originally mine and it looks like I don't have a copy. But the text tells you all you need to know about the hookup until I get off my butt and recreate a diagram of my own.

      Delete
  37. I have an issue with my ISS and am going to use your console and receiver to see if i can figure it out. Davis says to change the channel it is interference...im not convinced.

    BTW Davis is redirecting requests from your link to 404. Never saw a from-to on one of those before -- just sayin' ;-)

    What Happened?
    Error 404: The Page Cannot Be Found.

    Is this the page you were on?
    http://madscientistlabs.blogspot.com/2014/02/build-your-own-davis-weather-station_17.html


    Is this the link you tried to click?
    http://www.davisnet.com/support/weather/download/VantageSerialProtocolDocs_v230.pdf


    ReplyDelete
  38. Thanks for all of your work. I have a Vantage Pro2 with a custom cable, and FW 3.15. The serial is up, but only responds "NO" to everything. Have any ideas?

    ReplyDelete
    Replies
    1. Yup. The newer firmware looks for a newer Davis Datalogger and the serial port is locked down otherwise. But there are workarounds...

      http://meteo.annoyingdesigns.com/DavisSPI.pdf

      Delete
  39. Some months back I picked up a moteino and pop. it with clock,baro and hum. It's strapped to an rpi that also serves as OTA tv recorder. I am using cumulusmx to log and display data. Anyway, our first rain hit and I realized I had not addressed that in your code. As I operate on a 1 byte fifo, I have long since forgotten what all this is about. Could use a little help on the rain "issue". Would be appreciated but you already have done an awesome job. thanks!

    BTW: Should I concider using weewx instead of cumulus as the code is available for mod.

    ReplyDelete
  40. Hello Dekay,

    many thanks for your work. I bilding an Display with LED Segment and Dot-Display for better reading from far away.

    I had two Question to you.

    1. Did you have an Idea to use the RFM69 with an Arduino Mega 2560? I have change the cs, the Defines in *.h. But I don't get Date out the air, while the Arduino Nano and Uno funktion so far. (Not Stable, but I think thats the Artenna).

    Now I use two Arduinos witch communicate via Serial.

    2. Did you have an Idea to get Data from an second Wireless Station for the Soil und Extra Temperature in the Garden?

    ReplyDelete
  41. Hello my friend.
    I am Ruben from Spain.
    I am ordering now parts to build it.
    One little question:
    Can I use RFM69HCW?
    Tnx a lot

    ReplyDelete
  42. Hello Dekay
    Love the name handle.

    I'm venturing into IoT and am attempting to merge the RF69 Mega to a PI 3 as a weather input, and create a Cumulus like head for weather displays. The RF69 will read inputs from a Davis VP2 via wireless signals. I was wondering if you could update your pic on the hardware hookup.

    Thanks for your hard research on this. I don't think I'd attempted this without standing on your prior work.

    ReplyDelete
    Replies
    1. Hello Dekay
      Same anonymous person here, but I can fix that.
      My name is Kevin Cross

      I got sketchs loaded onto the Monteino Mega USB today
      Not picking up radio signals reception and serial transmission though via the serial port monitor. I'll look into that later. It probably has something to due with your changes not being tested against the MEGA, at least that's what I've gathered with the remarks.

      During setup I ran across this little item, http://www.airspayce.com/mikem/arduino/RF69/, regarding the RFM69 library and was wondering if you had any thoughts on it. Since you made some modes to the original, which I used in my MEGA, I thought maybe you might have had some recent experience. It certainly seems that it can serve up a lot more than RFM69.

      I'll plod forward on development and return from time to time to post new stuff I find, and share solutions when I have something to show.

      Bye for now.

      Delete
  43. Update: Making slow progress. Life gets in the way, but that's a good thing in my case.
    KCross

    ReplyDelete
  44. No postings on this board for sometime now. I hope life finds you in good health or otherwise a great setting.

    I'm retired now and have made better progress.
    I've got the RF69 Monteino Mega receiving data from the Davis ISS. I am now trying to integrate BME680 sensor from Adafruit for indoor Gas, Humidity, Temperature, and Barometric pressure.


    Next I'll clean up software and add it to a PI Prototype Hat that will run weewx software, and other services. Eventually I hope to have it communicate through the GPIO Serial RX/TX pins and avoid another wire.

    The Proto hat mounts the Monteino nicely, and has room for the RTC and BME680. The PI case allows the GPIO mount and accepts a PI 7" touch screen. This will be my display and allow me to switch functions for future development with Openhab home hub and integration with Alexa. That's my hope anyway.

    I'll get a blog or sourceforge/GitHub account after this phase and post my project. This page got me started and I've discovered other points of reference to here during developing.

    Thanks for the inspiration.

    Kevin Cross

    ReplyDelete
  45. Greetings
    For trial start and reading of these com ports it is better to use https://www.eltima.com/products/serial-port-monitor/. For your development, this is a worthy product.

    ReplyDelete
  46. This one looks interesting. Is this still/currently working with the very latest firmware versions? And do I understand you right, you managed to get rid of the original Davis Console? Would this in reverse mean one could save the money and buy the Davis ISS only wihtout Console?

    ReplyDelete
  47. What is the status of this project? Are there other projects out there that might supersede this one? I am acquiring the parts to experiment with this.

    ReplyDelete
  48. Please, Hardware hookup image is not available.
    Where could i see it?
    Thank's for the great project.

    ReplyDelete
  49. This comment has been removed by the author.

    ReplyDelete
  50. I can’t say I’ve seen hundreds of great posts on this particular topic, especially when it includes a special section like Weather Station Console.

    climbingreviewed

    ReplyDelete
  51. Hey, I read the post entirely and couldn’t help thanking you for sharing with us this piece of informative resource.
    reviews29

    ReplyDelete
  52. If believe the facts, then https://com-port-monitoring.com is much better at monitoring COM port data.

    ReplyDelete
  53. You possess lifted an essential offspring..Blesss for using..I would want to study better latest transactions from this blog..preserve posting.. weather station

    ReplyDelete
  54. This comment has been removed by the author.

    ReplyDelete
  55. Awesome! No words. You always go one step beyond.

    There is so much great, useful information here. Thank you! Thank you! Thank you!
    Read our guide if you wish.
    Besttoolsbrand

    Thanks again :)

    ReplyDelete
  56. Thanks for your work on this. On a Kobuki WX forum entry you indicated that you were working on an ESP8266/Motino combination. Did that go anywhere? Google got me nowhere.

    ReplyDelete
  57. It'S Science, But It Works Like Magic.: Build Your Own Davis Weather Station Console!!! >>>>> Download Now

    >>>>> Download Full

    It'S Science, But It Works Like Magic.: Build Your Own Davis Weather Station Console!!! >>>>> Download LINK

    >>>>> Download Now

    It'S Science, But It Works Like Magic.: Build Your Own Davis Weather Station Console!!! >>>>> Download Full

    >>>>> Download LINK

    ReplyDelete