After Resetting Frame Counters, Payload Is Shown On Gateway Traffic But Not in Application - Application Development - The Things Network

21/01/2020 After resetting frame counters, payload is shown on gateway traffic but not in application - Application Development - The
nt - The Things N…
After resetting frame counters, payload is shown on gateway traffic but not in
Nov 2018
application
11 / 16
Nov 2018
Sep 2019
https://www.thethingsnetwork.org/forum/t/after-resetting-frame-counters-payload-is-shown-on-gateway-traffic-but-not-in-application/19331/11 1/8
21/01/2020 After resetting frame counters, payload is shown on gateway traffic but not in application - Application Development - The Things N…
Nov 2018
11 / 16
Nov 2018
Sep 2019
Nov 2018
11 / 16
Nov 2018
Sep 2019
c_mourouzis Nov '18
Your MQTT code works for me; if I make the Payload Format do return 1/0; then, when simulating an uplink through the device in TTN Console,
I get:
Connected
Uplink Errors
Received uplink error from my-device-id
{ error: 'Unable to decode payload fields: Decoder not valid: does not return an object' }
It is also working for me, when simulating an uplink. But for real data I don’t get any uplink, downlink or activation errors. I don’t get anything.
Is there any way to find the reason why the message is not forwarded to the application? Is there a proper way to debug the data flow from the network
server to the application? It keeps sending data to the gateway and I can see the device address is still the same with the one on the application.
Does the trace information in the Traffic page in TNN Console look something like the following?
And is the Payload Format still okay, like “Custom”?
Exactly like that! I have also disabled the frame counter checks (last reply at Data received by the gateway but not on application) but that did not
make a difference. The device looks dead on the application console.
Just wondering if and how is possible to find the error log. The subscription to the MQTT API error events does not give any results. The data seem not
be forwarded at all to the application for some reason.
Once again, thank you for your help!
arjanvanb Arjan Nov '18
Seems like a backend issue indeed; I’ve pinged @htdvisser but surely the team is always busy. Nov 2018
Is the Handler for the application in TTN Console still okay? Something like the following in the application’s Overview:
Is the Collos integration available to be added again? (Just wondering if that has been removed completely, though if available that might not tell you
much.) Do you have any other OTAA devices with which you could test if one can remove the application and device and register new ones with the
same DevAddr and secrets?
As the trace in gateway’s Traffic page shows no errors, I’d assume that TTN has matched the DevAddr and NwkSKey to an11 / 16
application, so I’d guess
Nov 2018time.) But just in
these are still unchanged at the TTN side as well. (Surely the node is still using the same OTAA settings it has been using for a long
case: do you have any way to validate if the secret AppSKey and NwkSKey are still the same as before? (I guess you don’t.)
c_mourouzis:
Just wondering if and how is possible to find the error log. Sep 2019
No, not possible. You’ll need help from the team for that. Please post the gateway ID, application ID and DevAddr here. (These are not secret.)
Also, is a simulated uplink (though TTN Console’s Device’s page) shown in TTN Console’s Application Data page? And does ttnctl devices info
<device-id> show anything weird? See https://www.thethingsnetwork.org/docs/network/cli/quick-start.html
htdvisser Hylke Visser Core Team Nov '18
c_mourouzis:
shown in the gateway traffic with the same Device Address as before
Device Addresses are not unique. There are hundreds of devices with the same DevAddr, so I wouln’t be too sure about this.
c_mourouzis:
I tried to reset the frame counter
This may help for ABP devices, but for OTAA devices this could make things worse (as downlinks will be rejected by the device).
c_mourouzis:
Is there any way to find the reason why the message is not forwarded to the application? Is there a proper way to debug the data flow from the
network server to the application?
Unfortunately not. We’re working on this for v3.
arjanvanb:
Seems like a backend issue
Backend telemetry shows nothing out of the ordinary. If it were a backend problem we would see more than one user/device being impacted.
Unfortunately, even if we had time, it would be really difficult to trace individual messages with our current network stack, so I can’t help you with that.
The only thing I can say is that we haven’t made any changes to our servers, and that adding an integration has absolutely no influence on the uplink
flow of the device.
Nov 2018
c_mourouzis Nov '18
htdvisser:
c_mourouzis: 11 / 16
Nov 2018
I tried to reset the frame counter
This may help for ABP devices, but for OTAA devices this could make things worse (as downlinks will be rejected by the device).
Sep 2019
I disabled the frames check counter. Shouldn’t that have solved any frames counters issue? Also, if the device has ADR enabled it takes into
consideration the downlink counter? If this is the case I can manually set the up/down frames counter from CLI. Should the uplink counter be configured
as lower than the one seen on the gateway traffic and the downlink counter to a larger number than the one on the device (unknown but could be a
large number)?
arjanvanb:
Also, is a simulated uplink (though TTN Console’s Device’s page) shown in TTN Console’s Application Data page? And does ttnctl devices
info <device-id> show anything weird? See https://www.thethingsnetwork.org/docs/network/cli/quick-start.html
Yes the simulated uplink is shown on the console and captured by the MQTT Data API subscription too. Nothing weird in the device info!
arjanvanb:
As the trace in gateway’s Traffic page shows no errors, I’d assume that TTN has matched the DevAddr and NwkSKey to an application, so I’d
guess these are still unchanged at the TTN side as well. (Surely the node is still using the same OTAA settings it has been using for a long time.)
But just in case: do you have any way to validate if the secret AppSKey and NwkSKey are still the same as before? (I guess you don’t.)
It is not possible to assign a custom DevAddr anyway. This is assigned automatically by the TTN backend.
arjanvanb:
No, not possible. You’ll need help from the team for that. Please post the gateway ID, application ID and DevAddr here. (These are not secret.)
AppID
DevAddr
GatewayID
If the frames check counter isn’t the case, the network credentials and the application credentials have not changed why the payload does not arrive at
the console?
What are the possible issues? Why should the application server reject a payload when all the keys are correct? Why is not arriving at the application
server at all? If you can, please make this clear!
Thank you again! I guess I will have to go there and reset it
c_mourouzis:
I guess I will have to go there and reset it
I really doubt that will help. Resetting the node would make it execute a new OTAA join, but why would that help…? (Update: it likely will help; see next
Nov 2018
posts.)
c_mourouzis:
Why should the application server reject a payload when all the keys are correct? Why is not arriving at the application server at all? If you can,
please make this clear!
Indeed, I very much wonder about that as well.
c_mourouzis: 11 / 16
Nov 2018
I disabled the frames check counter. Shouldn’t that have solved any frames counters issue?
For TTN: yes maybe; see next post. For your node: no. The node does not know you changed that setting.
c_mourouzis: Sep 2019
Should the uplink counter be configured as lower than the one seen on the gateway traffic and the downlink counter to a larger number than the
one on the device (unknown but could be a large number)?
Yes. (But the uplink counter known to TTN will already have adjusted itself (see next posts). It’s only the downlink counter that might be troublesome for
ADR, as the value known to TTN will be lower than the one known in the node. But I’d assume all that does not affect your current problems. Doing a
new OTAA Join will fix all counters as well.)
c_mourouzis:
It is not possible to assign a custom DevAddr anyway.
Maybe the following will do the job; see also https://www.thethingsnetwork.org/docs/network/cli/api.html#ttnctl-devices-set
Can I register an ABP device that has fixed DevAddr, NwkSKey and AppSKey?
ttnctl devices set my-device-id --override --dev-addr ... --nwk-s-key ... --app-s-key ...
But: I’ve never tested that. (And the post above is about DevAddr that starts with 00.)
So, do you have any other devices to test the above?
c_mourouzis:
Hmmm, that is not “Exactly like that!” when comparing to my screenshot? But it seems the additional “bridge” is due to different type of gateways; my
screenshot was for a TTN Gateway and indeed for a Kerlink I see two more lines, referring to some “bridge”. So that’s probably not related to any
problems either.
(As an aside: my note about posting screenshots for terminal output surely also applies to posting the DevAddr and all please don’t post screenshots
for data that someone might need to copy; would you really expect the team to re-type the DevAddr, gateway ID and all from a screenshot…? But well,
they already said they cannot investigate any further )
arjanvanb:
As the trace in gateway’s Traffic page shows no errors, I’d assume that TTN has matched the DevAddr and NwkSKey to an application, so I’d
guess these are still unchanged at the TTN side as well.
You can actually check that: paste the full LoRaWAN packet from the Gateway Traffic into an online decoder and also provide the secret keys.
This will tell you if the NwkSKey is correct as then the MIC will validate assuming the uplink counter value does not exceed 65,536; if it does exceed
that value then ignore MIC validation errors. And regardless the MIC, if the AppSKey is correct, then this will give you an unencrypted application
payload that you can paste into the Payload Format decoder’s test input (if any) or decode yourself…
arjanvanb Arjan Nov 2018 Nov '18
For the sake of completeness:
arjanvanb:
But the uplink counter known to TTN will already have adjusted itself.
…actually: maybe not. If the difference between the device’s uplink counter and the zero value in TTN exceeded the maximum difference between
two subsequent received values (MAX_FCNT_GAP or 16,384), then maybe TTN cannot recover from that.
11 / 16
Novcounter
Or even if the code knows the counter has been reset and hence simply knows it can accept anything, then if the node’s uplink 2018 exceeds 65,536
(or maybe twice that value; 131,072) then it might not be able to find the matching MSB. In LoRaWAN 1.0.x, the value of FCnt only holds the 16 least-
significant bits (LSB) of the actual frame counter. But for a 32 bits frame counter still all 32 bits are used when calculating the MIC. So, the server needs
to guess or try the other 16 bits when validating the MIC. The server can use its own internal counters for a best guess. And due to the maximum
allowed gap between the last known value and current value, the server needs to try, and will try , only one additional value for the MSB. And that will
fail after resetting the counters, if the node’s uplink counter is large. Sep 2019
Even though you also already disabled the frame counter checks, TTN would then still be unable to find a matching MIC. I’ve tested by changing a
NwkSKey of an ABP node in TTN Console. This also automatically reset the frame counters ; even just changing “Frame Counter Checks” or its
description will also reset the counters! (Bug 746 .) So even after restoring the correct NwkSKey TTN was still unable to find a matching MIC. In the
trace part of the Gateway Traffic this showed the exact same thing that we saw for my earlier success and your failure. So, indeed: TTN will not report
if it cannot match the MIC.
So: do you know the last 32 bits uplink counter value? If not, you can use a Node.js script to brute-force the MSB and determine the full 32 bits counter:
/*
* Brute-forces finding a LoRaWAN uplink counter's 16 most-significant bits.
* This needs:
* npm install lora-packet
*/
const lorapacket = require('lora-packet');
const nwkSKey = new Buffer('7A47...', 'hex');
const uplink = new Buffer('4053...', 'hex');
const packet = lorapacket.fromWire(uplink);

console.log(packet.toString());
const msb = new Buffer(2);

for (let i = 0; i <= 0xFFFF; i++) {
console.log(`Trying: ${i}`);
msb.writeUInt16LE(i, 0);
if (lorapacket.verifyMIC(packet, nwkSKey, null, msb)) {
console.log(`Found MSB: 0x${('0000' + i.toString(16)).substr(-4)}`);
console.log(`32 bits FCnt: ${i << 16 | packet.getFCnt()}`);
break;
}
}
Meanwhile I’ve also enhanced the online decoder to brute-force the FCnt’s MSB. So, the online decoder will also give you the 32 bits FCnt if it can
find a valid MIC, even though only 16 bits are given in the LoRaWAN packet.
c_mourouzis:
I can manually set the up/down frames counter from CLI
Indeed, the following fixed my ABP node again:
ttnctl devices set my-device --fcnt-up 123456
…and then there is hope!
Although I’ve no idea how to determine the last known downlink counter… So I guess ADR is now no longer working for that node, until it’s made to do
a new OTAA Join.
ABP node stopped working -> Fixed by creating new ABP.. Why?
Wifi/BLE Paxcounter Project with ESP32 boards

Application ttn-handler-eu not working with gateway in US
c_mourouzis Nov '18
I would like to say a big thank you for your effort. For real, I’ve never seen a person so supportive in a forum.
With your detailed guidelines my problem got fixed. Also lora-packet is a great tool! Great job! In brief what I’ve done:
Nov 2018
Copied and pasted from TTN application the device’s Network Session Key to the lora-packet javascript code (const nwkSKey = new
Buffer(‘7A47…’, ‘hex’);).
Copied and pasted the physical payload from the gateway’s traffic to the same code (const uplink = new Buffer(‘4053…’, ‘hex’);).
Ran your code with NodeJS, found a valid MIC and got the 32bitsFcnt (You were right. It exceeded 65,536 and for that reason the server
couldn’t validate MIC)
Then set it via ttnctl: ttnctl devices set my-device --fcnt-up 32bitsFCnt
My device is live again! 11 / 16

Nov 2018
Summary: Always be careful when resetting the frames check counter. If for any reason you can see your payload to the gateway traffic but not on your
TTN Application Console a) make sure you have the same secrets as before, b) find the 32-bit FCnt and set it to your device via ttnctl!
arjanvanb: Sep 2019

Although I’ve no idea how to determine the last known downlink counter… So I guess ADR is now no longer working for that node, until it’s made
to do a new OTAA Join.
Since the device is not moving I do not need ADR at all. So no problem for the downlink counter!
c_mourouzis:
Since the device is not moving I do not need ADR at all
Actually, you should not use ADR for devices that do move. Devices that are in a fixed location can use ADR to adapt to the changing network around
them, like when gateways are added or removed. ADR is too slow (it needs too many uplinks and all) to be used for moving devices.
EdanPotter Aug '19
Hello there. I’m with the same problem, any news? Thanks.
BoRRoZ Sep '19
Hi,
please supply more info (router, gateway ect) and describe YOUR problem… tnx
or maybe it’s more related to this topic ?
The NOT CONNECTED / NOT SEEN in console CENTRAL topic part 1
seems that more users experiencing these issues lately : gateway is working and data is coming through, but the map shows not connected. this
topic is to collect those problems I merged some older related post too please specify your gateway type and region!

After Resetting Frame Counters, Payload Is Shown On Gateway Traffic But Not in Application - Application Development - The Things Network

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

After Resetting Frame Counters, Payload Is Shown On Gateway Traffic But Not in Application - Application Development - The Things Network

Uploaded by

Copyright:

Available Formats

21/01/2020 After resetting frame counters, payload is shown on gateway traffic but not in application - Application Development - The

c_mourouzis Nov '18

And is the Payload Format still okay, like “Custom”?

Once again, thank you for your help!

arjanvanb Arjan Nov '18

arjanvanb Arjan Nov '18

htdvisser Hylke Visser Core Team Nov '18

I tried to reset the frame counter

Unfortunately not. We’re working on this for v3.

Seems like a backend issue

c_mourouzis Nov '18

Thank you again! I guess I will have to go there and reset it

arjanvanb Arjan Nov '18

I guess I will have to go there and reset it

Indeed, I very much wonder about that as well.

c_mourouzis: Sep 2019

It is not possible to assign a custom DevAddr anyway.

Maybe the following will do the job; see also https://www.thethingsnetwork.org/docs/network/cli/api.html#ttnctl-devices-set

So, do you have any other devices to test the above?

arjanvanb Arjan Nov '18

arjanvanb Arjan Nov 2018 Nov '18

For the sake of completeness:

const packet = lorapacket.fromWire(uplink);

const msb = new Buffer(2);

I can manually set the up/down frames counter from CLI

Indeed, the following fixed my ABP node again:

ttnctl devices set my-device --fcnt-up 123456

…and then there is hope!

Wifi/BLE Paxcounter Project with ESP32 boards

c_mourouzis Nov '18

My device is live again! 11 / 16

arjanvanb: Sep 2019

arjanvanb Arjan Nov '18

Since the device is not moving I do not need ADR at all

EdanPotter Aug '19

BoRRoZ Sep '19

or maybe it’s more related to this topic ?

The NOT CONNECTED / NOT SEEN in console CENTRAL topic part 1

You might also like