General Discussion

[jepsonrob]

I'm trying to get this to work with some smart bulbs and light strip, but currently the only missing piece for me is how you got the prefix & suffix in requests.json.
Are they just the packet header/footer or is there more to it than that?

Awesome work here by the way, thanks for all your work!

[codetheweb]

Yep. To find the prefix and suffix for new devices, you'll have to fire up Wireshark and sniff the packets going to your device. The data portion of the packet starts with a version number (probably 3.1) and ends with = or == (a base 64 encoded string). The rest of the packet before and after (not including other TCP segments, like destination IP, checksum, etc.) that data string is what's plugged into the prefixes and suffixes.

I'm guessing this padding has an actual meaning, but so far I haven't been able to find anything. @blackrozes has suggested that it's just random data with a few special bytes in the beginning and end as devices seem to expect a fixed packet length. If you have any theories, I'd love to hear them.

[jepsonrob]

Yeah I've just checked in Wireshark and it fits your description perfectly.

Are you sure everything before the 00 00 55 aa part of the prefix is necessary to use with on & off? It appears be the TCP information you were talking about, and you cold be hardcoding your own private IP addresses into it (looks like your outlet is on 192.168.0.108).

As to theories, I'm noticing that the 8th hex value (starting from 00 00 55 aa) increments by 1 with every request and everything else but the very last bit is identical across the board, including in my own packets. The last bit before the data appears to only be either b3, 9b, 46 or 00 in both my own and your prefixes. Not sure what these represent at all!

[codetheweb]

Oh wow thanks. I was going to say "sorry for the confusion, I meant that you shouldn't include the TCP packet metadata", but then I looked at requests.json 🙄. Oops. I'll push an update in a few minutes removing the hex metadata.

[codetheweb]

Updated with b766439. I've now whittled them down to this:

"outlet": {
    "status": {
      "prefix": "000055aa000000000000000a00000046",
      "command": {"gwId": "", "devId": ""},
      "suffix": "000000000000aa55"
    },
    "on": {
      "prefix": "000055aa00000000000000070000009b",
      "command": {"devId": "", "dps": {"1": true}, "uid": "", "t": ""},
      "suffix": "000000000000aa55"
    },
    "off": {
      "prefix": "000055aa0000000000000007000000b3",
      "command": {"devId": "", "dps": {"1": false}, "uid": "", "t": ""},
      "suffix": "000000000000aa55"
    }
  }

I tried tearing out the ...7... and ...a..., replacing them with 0s, but it didn't work. So they must mean something. Same with 46, 9b, b3.

Is it working with your device yet?

[fusionedv]

I think the devices are expecting a special packet size. In my script I create a buffer with this size and put the data inside like this:

var buf1 = new Buffer(171); 
buf1.write('000055aa00000000000000070000009b'+ (data),0,"hex");
buf1.write('0000aa55',167,"hex");

[jepsonrob]

Okay I've done some analysis of these packets and have figured some stuff out:

Prefix & Suffix
The prefix is always (from my tests) 16 bytes long.

The first half - 8 bytes 00:00:55:aa:00:00:00:6b - of our packet is static, except for the last (8th) byte. The 8th byte increments by one every time a new command is sent, but I've seen some results that add more than this using commands that were not on/off. At any rate, it is just a counter and I think it doesn't really matter what the value is.

The second half - 00:00:00:07:00:00:00:9b - gives 2 pieces of information: the type of packet being sent and the length of the remaining the data/packet. So far I've found that the 4th byte is 07 when sending commands, 08 when receiving a reply from the device, '9e' for broadcast messages, and 0a when getting the status (haven't tested the status flag but I'm assuming this is the case from your prefix in requests.json).

The final value (the 16th byte of the whole prefix) is the size in bytes of the subsequent packet (obviously as a hex value). This is the length of the encrypted data in bytes plus the suffix - basically everything after the prefix. From my tests the suffix has always been 8 bytes.

I have no idea how the suffix is created, but it doesn't seem to change anything and I've had good results leaving it as 000000000000aa55. I'm still not quite there with getting this working on these devices yet, but knowing how the prefix work feels like most of the heavy lifting.

Connections & Broadcasts

When the device is not connected, it sends out UDP broadcast packets to the network every 3-6 seconds. This contains plaintext JSON with the following information:
{"ip":"192.168.0.xx","gwId":"002003595ccfxxxxxxxx","active":2,"ability":0,"mode":0,"encrypt":true,"productKey":"AqHUMdcbxxxxxxxx","version":"3.1"}

This could be used as a way of enumerating devices on the network and all information other than the key.

Furthering the enumeration aspect, we get a plaintext list of commands available on the device from the initial connection handshake: sending the
{"gwId":"002003595ccxxxxxxx","devId":"002003595ccfxxxxxxxx"}
packet leads to a plaintext JSON status response from the device:
{"devId":"002003595ccfxxxxxxxx","dps":{"1":true,"2":"colour","3":255,"4":255,"5":"00ff0d007bffff","6":"00ff0000000000","7":"ffff500100ff00","8":"ffff8003ff000000ff000000ff000000000000000000","9":"ffff5001ff0000","10":"ffff0505ff000000ff00ffff00ff00ff0000ff000000"}}
which can in turn be used to enumerate JSON for the specific device.

One thing I've struggled with is actually decrypting the packets sent/received using the key. Has anyone got a deciphering script?

[codetheweb]

@jepsonrob thanks, that's really helpful.

Receiving Broadcasted Messages

For anyone interested, you can receive the UDP broadcasted packets with a simple script:

const dgram = require('dgram');
const server = dgram.createSocket('udp4');

server.on('error', (err) => {
  console.log(`server error:\n${err.stack}`);
  server.close();
});

server.on('message', (msg, rinfo) => {
  console.log(`server got: ${msg} from ${rinfo.address}:${rinfo.port}`);
});

server.on('listening', () => {
  const address = server.address();
  console.log(`server listening ${address.address}:${address.port}`);
});

server.bind(6666);

I may add auto discovery functionality soon, although it's not a huge priority for me so if some else wants to do it and open a pull request I'd be very grateful.

For Decrypting:

Basically, just do the reverse of this:

  // Encrypt data
  this.cipher.start({iv: ''});
  this.cipher.update(forge.util.createBuffer(JSON.stringify(thisRequest.command), 'utf8'));
  this.cipher.finish();

  // Encode binary data to Base64
  const data = forge.util.encode64(this.cipher.output.data);

  // Create MD5 signature
  const preMd5String = 'data=' + data + '||lpv=' + this.version + '||' + this.key;
  const md5hash = forge.md.md5.create().update(preMd5String).digest().toHex();
  const md5 = md5hash.toString().toLowerCase().substr(8, 16);

  // Create byte buffer from hex data
  const thisData = Buffer.from(this.version + md5 + data);
  const buffer = Buffer.from(thisRequest.prefix + thisData.toString('hex') + thisRequest.suffix, 'hex');

I'll try to add a specific decryption function within the next couple days.

[clach04]

@jepsonrob when you say, " not connected, it sends out UDP broadcast". Do you mean on the network but no devices are connected to it?

I tried both @codetheweb 's suggested js listener script and a Python script based on SSDP discovery code on port 6666 and I'm not seeing anything.

BTW thanks for excellent write ups, I now have a Python version, its rough and ready but works :-) https://github.com/clach04/python-tuya

One thing I've noticed is that if there are spaces in the json payload, the device will not respond. Not an issue under js, but the Python stdlib library adds spaces.

[jepsonrob]

@clach04 - that's exactly what I mean, yep. It's potentially useful for enumeration because all devices on the network can see these packets in plaintext and they contain a bunch of useful information.

And your issue with the JSON payload & spaces might come from the last 2 characters in the prefix - it needs to be the total length of the rest of the payload in hexadecimal. Using spaces will increase the size of the payload and if the value isn't the correct size then the command goes ignored.

[clach04]

@jepsonrob thanks for the length info, that's it for sure. I added support for this in clach04/python-tuya@fc4612f - I'm not sure if its a single byte for length or not (I've only coded it for a single byte and I've not yet added sanity check for that in case the payload goes above 255 bytes).

Thanks also for confirming about broadcast. Sadly, as per my previous comment, I'm not seeing this on my network (using code above i.e. something sitting on port 6666, I've not tried wiresharking the network). Any pointers?

I've created a wiki https://github.com/clach04/python-tuya/wiki to store progress and other useful related docs. E.g. I figured out the timer for the SM-PW701U device.

[jepsonrob]

@clach04 - try using this python script and see if anything comes through: it's working for me!

from socket import *
s=socket(AF_INET, SOCK_DGRAM)
s.bind(('',6666))
m=s.recvfrom(1024)
print m[0] 

And yeah I've only had it working for a single byte which is driving me crazy because the payload I need is over 255. Interestingly though, when I'm looking at the packets sent in Wireshark there's never anything over 255 bytes in size! I feel like this means I'm getting something wrong with my JSON payload and it's coming in too large, but I can't quite figure out how to decrypt the outgoing packets so I can't see the intended behaviour.

Nice work on the python version by the way! That wiki is a useful resource - I'm going to write a no-stupid-questions type of tutorial for this we've got it working with other devices, probably just to help people with IFTTT integration, and I'll link out to that for further reading.

[clach04]

@jepsonrob that script worked fine. My script also worked fine this morning (not sure why, maybe too tired last night).

Interesting notes:

• I'm testing on Windows at the moment
• python script only works if I'm on the WiFi network, being on the same network via a wire doesn't work (I thought I tested this last night but maybe was not testing with py2) - EDIT looks like this was a Windows firewall issue, specific Python process blocked on certain networks
• python script only works for me with Python 2 (change print to add parens for py3), under Python 3 I do not see any packets - this one still confuses me. Not sure if this my machine, Windows, etc. - EDIT looks like this was a Windows firewall issue, specific Python process blocked on certain networks
• node script above does NOT work for me (possible same root cause as py3 failure?) - EDIT gonna guess this was a firewall issue too

@jepsonrob script (slightly modified):

from socket import socket, AF_INET, SOCK_DGRAM

s = socket(AF_INET, SOCK_DGRAM)
s.bind(('', 6666))
m = s.recvfrom(1024)
print(repr(m[0]))

My script:

# discover devices
# NOTE for my devices this only works with Python 2.6
# py3.6.1 runs but never reports packets

import socket
import struct


host_port = 6666
host_ip = '239.255.255.250'

sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, socket.IPPROTO_UDP)
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
sock.setsockopt(socket.IPPROTO_IP, socket.IP_MULTICAST_TTL, 2)
sock.bind(('', host_port))

mreq = struct.pack('4sl', socket.inet_aton(host_ip), socket.INADDR_ANY)
sock.setsockopt(socket.IPPROTO_IP, socket.IP_ADD_MEMBERSHIP, mreq)

try:
    while True:
        print('listening')
        data = sock.recvfrom(1024)
        raw_bytes, peer_info = data
        print(data)
finally:
    sock.close()

[clach04]

@jepsonrob for decrypting packets, try this (needs my Python module, https://github.com/clach04/python-tuya):

import base64
import time

import pytuya


key = 'YOUR_KEY_HERE'
packet = 'Python raw bytes'


PREFIX_LEN = 16
SUFFIX_LEN = 8
prefix = packet[:PREFIX_LEN]
json_crypted_plus_suffix = packet[PREFIX_LEN:]
json_crypted_len = ord(prefix[-1])
json_crypted_b64_with_header = json_crypted_plus_suffix[:json_crypted_len-SUFFIX_LEN]
version_str = json_crypted_b64_with_header[:3]
print('version_str (%d) %r' % (len(version_str), version_str))
hexdigest = json_crypted_b64_with_header[3:][:16]
print('hexdigest (%d) %r' % (len(hexdigest), hexdigest))
json_crypted_b64 = json_crypted_b64_with_header[3+16:]
suffix = packet[-SUFFIX_LEN:]  # assume we know length of end


print('packet (%d) %r' % (len(packet), packet))
print('prefix (%d) %r' % (len(prefix), prefix))
print('json_crypted_plus_suffix (%d) %r' % (len(json_crypted_plus_suffix), json_crypted_plus_suffix))
print('json_crypted_len %d' % (json_crypted_len))
print('suffix (%d) %r' % (len(suffix), suffix))

print('json_crypted_b64 (%d) %r' % (len(json_crypted_b64), json_crypted_b64))
cipher = pytuya.AESCipher(key)
json_raw = cipher.decrypt(json_crypted_b64)
print('json_raw (%d) %r' % (len(json_raw), json_raw))

EDIT only supports 255 lengths (could be updated if it turns out protocol supports it.