1. Carbon Series
Sign up or log in for full site functionality and to hide this notice.

Join to the community in order to comment, like, post images, videos and create threads.

:)

Evolve - radio is using nRF24L01

Discussion in 'Carbon GT' started by julian46, Nov 26, 2016.

More threads by julian46
  1. OP
    julian46

    julian46 Member

    Joined:
    Nov 14, 2016
    Messages:
    251
    Likes Received:
    101
    Location:
    Toronto, ON, Canada
    Real Name:
    Julian
    Boards Owned:
    Bamboo GT
    Wheel Setup:
    Street
    Wheel Size:
    83
    Map
    nope - not bluetooth - these hackers checked it out:

    Interesting - with previous gen board (non GT - 2015) they were using a similar chip set - nRF24LE

    they couldn't hack the remote (not bluetooth - so that's an upside ?)

    say its using ShockBurst (a simplex protocol)
    - this previous gen says "no data to the remote" - but since the nRF24L01 in our config is a transceiver both ends (the remote and the board can send and receive - but take turns one direction at a time)
    - our boards have "Enhanced ShockBurstT" - not sure whats improved

    defcon-23-evolve-1.jpg


    looks like in the last 2-3 years there have been 3 versions of the remote

    pic of old old style (non GT) remote
    evolve-electric-skateboard-10-old-remote.jpg

    pic of old (just pre GT) remote
    evolve-pre-gt-remote.jpg

    Its an interesting video below and worth a full watch - the TL;DR summary with Evolve is that they couldn't do sophisticated things with it apart from jamming the signal (it ran down a hill - when the board lost the remote) - contrast this with boosted where they could actually execute functions - so safer non bluetooth ? (maybe in some ways)

    go to 24:16 for Evolve section:
    printed slide summary here:
    https://evolveforums.com/attachment...0/?temp_hash=d8c6d2738958ef5c337309ed03c883d6

    DefCon23-2015-ElectricSkateboardJamming





    a 2year old video showing the second gen (non GT remote)



    from this subred 5 months ago (it had high power mode ??):
     

    Attached Files:

    Last edited: Dec 2, 2016
  2. OP
    julian46

    julian46 Member

    Joined:
    Nov 14, 2016
    Messages:
    251
    Likes Received:
    101
    Location:
    Toronto, ON, Canada
    Real Name:
    Julian
    Boards Owned:
    Bamboo GT
    Wheel Setup:
    Street
    Wheel Size:
    83
    Map
    the nRF24L01 also seems to be used a fair bit in the DIY electric skateboard community
    I've found several hits where its paired with a common open source ESC (electronic speed controller) - the VESC

    I've posted replies on bunch of threads and comments on videos where I've seen the nRF24L01 - will post any useful feedback comments in this thread.

    Vedder VESC – Open Source ESC
    VESC – Open Source ESC | Benjamin's robotics

    Note this thread about problems and warnings with specific nRF chips:
    VESC NRF NUNCHUK WARNING! - Page 2 - vedder.se forums

    CHECK out this first post in this thread below - sound familiar ???
    VESC NRF NUNCHUK WARNING! - vedder.se forums
    VESC NRF NUNCHUK WARNING!

    --------- quote ---------
    Hey guys,

    just want you to know that it is dangerous (my opinion) to use the NRF nunchuk right now. Today I got a disconnect which was resulting in a unstoppable full speed. No chance to reconnect over one kilometer with full throttle (looks like the safety feature to turn off was not working) ended into a 52 kmh crash. Not good. [​IMG]
    I hope to find out where this fault came from. After a battery reconnect the nunchuk is working as before.

    =====================

    NOTE - our Evolve GT's are not using this specific ESC or using the same controller code (per se) - it was more the comment on the RF link / nRF radio control I was interested in....

    On page 2 - of that NRF NUNCHUCK warning thread they go into great detail on possible causes (including the esc dev) - worth looking at.

    =====================

    This section is especially interesting (again page 2) - see what the dev (Benjamin) says in the last line of this paragraph

    --------- quote ---------
    I'm sorry to hear that this happened. There is a timeout thread in the code that will stop the motor when no command has been received for more than a configurable time (1000ms default). A configurable brake current can also be applied when the timeout occurs. Both the timeout and brake current can be configured in BLDC Tool. I had a careful look at the timeout source code and found one bug: When running flux linkage detection for FOC, the timeout is disabled temporally and not enabled again until after a reboot or the app configuration is set. Another possible way that this could happen is that the NRF chip gets into some weird state so that it always reports that a packet is in the rx buffer and the same packet is read out over and over again.

    =====================

    theres more gems on that site - look at this: NRF Nunchuck Signal drop/weird behavior ! - vedder.se forums
    - diy two way remote (control and telemetry similar to ours) - Arduino nRF two way comms: Current progress! - vedder.se forums
     
    Last edited: Dec 2, 2016
    • Like Like x 1
  3. Fresno

    Fresno Member

    Joined:
    Nov 10, 2016
    Messages:
    22
    Likes Received:
    12
    Location:
    Fresno, California, US
    Real Name:
    Ron
    Wheel Setup:
    All Terrain (AT)
    Map
    Julian, when you start making your own boards, sign me up for two.
     
  4. OP
    julian46

    julian46 Member

    Joined:
    Nov 14, 2016
    Messages:
    251
    Likes Received:
    101
    Location:
    Toronto, ON, Canada
    Real Name:
    Julian
    Boards Owned:
    Bamboo GT
    Wheel Setup:
    Street
    Wheel Size:
    83
    Map
    LMAO - haha - we will make it a team effort - Wiztecy just found another source for the motors - pretty sure before too long we could actually build our own ! - but with a better remote :)
     
  5. forbesmyester

    forbesmyester Member

    Joined:
    Oct 31, 2016
    Messages:
    79
    Likes Received:
    27
    Location:
    Ware, United Kingdom
    Real Name:
    Matt F
    Boards Owned:
    Bustin Mission (Gen 2)
    Wheel Setup:
    Street
    Map
    Julian46, of course the chip has an RX buffer, it's be pretty useless without one.

    Of course it could theoretically get in a strange state where it keeps reporting that RX buffer. That is true of any chip that reads any buffer. Why are you telling us this?

    I think we should wait for Covert to come back to us and let us know what he's found out.
     
    Last edited: Dec 3, 2016
  6. OP
    julian46

    julian46 Member

    Joined:
    Nov 14, 2016
    Messages:
    251
    Likes Received:
    101
    Location:
    Toronto, ON, Canada
    Real Name:
    Julian
    Boards Owned:
    Bamboo GT
    Wheel Setup:
    Street
    Wheel Size:
    83
    Map
    forbemyester, I posted this because not everyone knows the chip has an RX buffer (receive buffer) or even know what it means - do you not find it odd that it can get stuck in that state ? - where else have you seen this happen - this is not a normal condition - there should be code to account for that or it could be a batch of poor quality chips causing this (that need to be replaced)

    if you have anything useful to add - please do so - open to other explanations you may have - one of the reasons for this thread is to try and figure out what happened and help eliminate it - its not like the problem is a "one off", its common enough now to compare experiences and research other installations of the nRF chipset where they may also be having issues.
     
    Last edited: Dec 3, 2016
  7. forbesmyester

    forbesmyester Member

    Joined:
    Oct 31, 2016
    Messages:
    79
    Likes Received:
    27
    Location:
    Ware, United Kingdom
    Real Name:
    Matt F
    Boards Owned:
    Bustin Mission (Gen 2)
    Wheel Setup:
    Street
    Map
    Why do you think "code" clears the buffer?

    I think you've got a lot of debugging to do before you start cracking open the chip schematics...

    You're getting all excited over almost certainly silicon level bugs in specific chips we likely don't have.

    I'm not saying it is not a problem tho.
     
  8. OP
    julian46

    julian46 Member

    Joined:
    Nov 14, 2016
    Messages:
    251
    Likes Received:
    101
    Location:
    Toronto, ON, Canada
    Real Name:
    Julian
    Boards Owned:
    Bamboo GT
    Wheel Setup:
    Street
    Wheel Size:
    83
    Map
    so whats your point - we just have to live with it ?

    no plans to start opening chip schematics

    why do you think we don't have these chips ?

    if you have a theory or strategy going forward lets hear it
     
  9. forbesmyester

    forbesmyester Member

    Joined:
    Oct 31, 2016
    Messages:
    79
    Likes Received:
    27
    Location:
    Ware, United Kingdom
    Real Name:
    Matt F
    Boards Owned:
    Bustin Mission (Gen 2)
    Wheel Setup:
    Street
    Map
    Any issues, in any tech, are likely to be config, unexpected input, code related to unexpected input (in our case interference), code (non input related) or silicon... Pretty much exponentially less likely in that order.

    It's a bit different in embedded, and you think about it differently too but you still don't start at the silicon level. You work your way down from the top.

    I don't think it is likely that we have those chips as unless someone tells me different it sounds like commodity hardware to me. They are quite probably from many manufacturers and with many iterations / slight variations. Any problems have probably been fixed already, likely before Evolve even existed.

    Don't go scaring people about scary things like RX buffers which must, of course be evil, if only we could remove them... Any problem is likely to be something much, much, much more simple, that's not to say it is simple to find, they're always where you least expect!
     
  10. wiztecy

    wiztecy Member

    Joined:
    Oct 13, 2016
    Messages:
    621
    Likes Received:
    419
    Location:
    Santa Cruz, CA, United States
    Real Name:
    Dennis
    Boards Owned:
    Carbon GT, Other Electric Skateboard
    Wheel Setup:
    Street & AT
    Map
    As a person who works on embedded systems as a career, its not at all odd finding bugs that the manufacturer of a CPU / controller / etc in a well known manufacturer's product. Many times they know about these issues, have written errata's about them, but its only until you have issues and contact them will they let you know there's an issue in their piece of simple logical silicone hardware product. And there are also times where they don't know about it and you're the first one to bring it to their attention. Its true to start at the top software layer and work your way down, however, from much of my debugging there's this crazy misunderstanding people get into thinking that hardware has no bugs! That's insane and completely wrong! Work in the industry for some time to discover the reality of that biasness. Also there's a serious disconnect between software/hardware engineers in the industry, they don't talk the same language so many issues arise due to this communication deficiencies. As a naive customer, we have also this assumption that's wrong that software has no bugs let alone hardware. Consumers have no grasp of the complexity of proper software / hardware testing and really put their life on the code / hardware. It was only until I started my career 16+ years ago to find how flawed both of these "wares" are and is only as good as the blackbox, whitebox, greybox and adhoc testing coverage holds and discloses the real truths of the product.

    Coming up with theories as to why something does not work the way it should IS part of the real investigative processes. Yes, reality is that some of these theories will scare you and people, but you cannot ignore those theories for that they may be valid. Its only as you dive deeper and build up more concrete information that you throw away the theories that do not fit and you begin narrowing down the true inputs / outputs and unknowns that were not found and ignored of the real issue that were originality missed by you or the company who produced the product.
     
    Last edited: Dec 3, 2016
    • Informative Informative x 1
  11. OP
    julian46

    julian46 Member

    Joined:
    Nov 14, 2016
    Messages:
    251
    Likes Received:
    101
    Location:
    Toronto, ON, Canada
    Real Name:
    Julian
    Boards Owned:
    Bamboo GT
    Wheel Setup:
    Street
    Wheel Size:
    83
    Map
    forbesmyester I don't disagree - you raise some good points - its just the chance of us getting high level information is pretty low - I don't think there is any way Evolve would "come clean" or explain what happened and how it was fixed - we will more likely just see it go away with hardware swaps.

    Of course if this takes a long time then it may have required a code fix that took some time to troubleshoot under the exact conditions that produce it in the first place. - my two main theories continue to be "suspect code" in the latest gear-up / gear-down control firmware or simply a bad batch of YJ-13039 boards meaning suspect or less stable nRF24L01 chips - again see the link to the thread in previous pages where tech types spend a whole bunch of time identifying the original Nordic Semiconductor chips from the counterfeit ones (potentially less reliable)

    I am still looking for a really good clear picture of the nRF chip on our boards to help id the actual batch / die / source - the boards for sure are YJ-13039 and contain these chips.

    and for sure a big bunch of this is speculation but that's what started this thread in the first place to see what we are dealing with and what makes up the radio system - always open to other theories, suggestions, ideas etc.
     
    • Like Like x 1
  12. Alex

    Alex Admin

    Joined:
    Sep 9, 2016
    Messages:
    1,554
    Likes Received:
    967
    Location:
    Barnes, London, United Kingdom
    Real Name:
    Alex Jermstad
    Boards Owned:
    Carbon
    Wheel Setup:
    All Terrain (AT)
    Map
    Just saw this about the connectivity on MellowBoards new board.

     
    • Like Like x 2
  13. feannorr

    feannorr Member

    Joined:
    Jan 12, 2017
    Messages:
    74
    Likes Received:
    34
    Location:
    Rathfarnham, Dublin, Ireland
    Real Name:
    James
    Boards Owned:
    Bamboo GT
    Wheel Setup:
    Street
    Map
    Very interesting discussions

    I would firstly declare several points - firstly I have my boards for almost 2 months now, a bamboo GT - and have experienced no issues whatsoever with my remote - no 'drop outs' - no unexpected braking - no powered movement without input.

    Secondly I am an electronic engineer - and also have degree in computer science - so my expertise in in both the 'hardware' and 'software'

    From what i interpret from a lot of the descriptions of the issue is as follows: (an please correct me if I am wrong)
    1. 'drop outs' - these involve moments when the board refuses to react to the commands from the remote
    2. emergency stops - where the board will apply full brake unexpectantly - usually occurs after a 'drop out'

    I have also looked through the specification of the shock burst protocol, which seems a modern robust comms protocol - including full ACK and 2 way comms, and wide frequency range and channel selections.

    I would consider the issue to have several possible causes:
    1 - fault rx / tx hardware chipsets - could also involve power issues with interruptions or current drops....this could be at either end. The remote itself or the board.
    2 - software problems - unexpected loops and timeouts
    3 - combinations of the above.

    The drop out seems to indicate that the remote is queuing up its commands and sending them in sequence to the board, there would only ever be one command in that queue if everything is working correctly - however the shock burst protocol requires acknowledgements from the receiver - so either the board if failing to respond and send an acknowledgement to a received command or the remote is failing to transmit the command. In either event the software in the remote queues up commands - and continues to resend them in sequence until it gets an acknowledgement. (there would have to also be timeout set to tell the remote to drop the queue)

    The point of failure then can be many different places - from remote transmitter circuit - to board receiver circuit - to the remote logic controlling the transmitter - to the board logic controlling its receiver.

    What is needed is a test procedure to use a process of elimination - and each individual case may be different.

    I would suggest making a basic faraday cage - then pair a remote to a board - then place the remote in the faraday cage (to break the radio contact between the devices) - then make several commands in a sequence - then remove the remote and observe whether those commands are then carried out by the board in the correct sequence (albeit delayed by the radio signal break)

    Once established repeat - increasing the time after sending blocked commands and removing from faraday cage - the result should show you what the default timeout is for repeated commands.

    Make sense?

    .
     
  14. feannorr

    feannorr Member

    Joined:
    Jan 12, 2017
    Messages:
    74
    Likes Received:
    34
    Location:
    Rathfarnham, Dublin, Ireland
    Real Name:
    James
    Boards Owned:
    Bamboo GT
    Wheel Setup:
    Street
    Map
    I would add one other experiment to be tried........

    Pair remote and board - the accelerate to max - observe wheels spinning under power - then place the remote into the faraday cage will still set to accelerate - observe results of the communication link being broken........

    Use different times in faraday cage and check if a timeout results in an emergency stop - or results in continued power.

    Hit brakes several times - remove from faraday cage and observe behaviour.....

    Should shed some light on the issue
     
  15. OP
    julian46

    julian46 Member

    Joined:
    Nov 14, 2016
    Messages:
    251
    Likes Received:
    101
    Location:
    Toronto, ON, Canada
    Real Name:
    Julian
    Boards Owned:
    Bamboo GT
    Wheel Setup:
    Street
    Wheel Size:
    83
    Map
    yea I think your notes above and summary is correct - QUEUING up is the best way to describe what happens

    So you get the drop out (failure in communication not matter what the cause)

    While the rf breakdown is taking place, any further inputs from there on are ignored (and maybe as you say until the ACK is taken care of and things start to flow again)

    This could happen while accelerating, while braking or even changing gear (speed modes) - i think its all delivered down the same pipe

    Then - things start to flow again and the board DOES everything you just told it to do in sequence (and if you were braking hard while it was ignoring you, eventually it completes the commands and by this time you have already applied too much brake so the board stops dead)

    We have posted a bunch of theories on this thread about what may be the exact cause (and there could be multiple) - from software bugs to bad hardware and poor quality NRF clone chips.

    I think if Evolve focused on simply the slope of the ramp up (acceleration) and ramp down (deceleration) and programmed the motor controller to not make such sudden changes we would at least have something safer. - or add an addition "Sport Mode" that has steeper slopes and also enables reverse - which for basic cruising you never would need)

    My first choice would be for them to switch to real Bluetooth which has frequency hopping and encryption built in - we don't even know if they have implemented that on the NRF24L01 transceivers - I some how doubt they have. (there have been reports of two boards being controlled by one remote on some of the group rides - would this happen with encryption (?) , also my guess is the pairing process picks a new channel so frequency hopping also is likely not implemented)

    Real Bluetooth would also give them the option to enhance the app and allow field updates of the firmware (so nothing has to be sent back to Evolve service to be improved with upgraded features and fixes)
     
  16. feannorr

    feannorr Member

    Joined:
    Jan 12, 2017
    Messages:
    74
    Likes Received:
    34
    Location:
    Rathfarnham, Dublin, Ireland
    Real Name:
    James
    Boards Owned:
    Bamboo GT
    Wheel Setup:
    Street
    Map
    julian46

    I would agree with you - and from the spec the shock burst protocol is a Ack based system - with built in retransmission and proof of delivery - similar to TCP. So it will continue to reattempt transmission until it receives the ACK.

    You are correct also - the cause of communications break - could vary between individual cases - but the result is the same - queued calls eventually acted on when you are not prepared for them.

    My recommendation would be to not brake hard when a drop out occurs -- or randomly and strongly vary the command from the remote - this will only make the final action more violent from the board. Best to be aware and if a command is not acted on......then patiently wait until it is......easier said than done, but only break again after this has happened....would be the safest mitigation.

    As for Evolve - my solution for them is to include additional logic and software on the receiver side (the board).

    This should monitor for communication dropouts - and if detected, a gradual braking procedure should automatically be activated, bring the board to a full stop gradually and safely. Then require a full repairing before it can be used again. Also a logging function within the software for error correction, and reporting to evolve - for continual improvement.
     
    • Like Like x 1
    • Agree Agree x 1
  17. feannorr

    feannorr Member

    Joined:
    Jan 12, 2017
    Messages:
    74
    Likes Received:
    34
    Location:
    Rathfarnham, Dublin, Ireland
    Real Name:
    James
    Boards Owned:
    Bamboo GT
    Wheel Setup:
    Street
    Map
    Personally speaking the shock burst method looks excellent to me, and possibly better than bluetooth, certainly higher power.....which is what you would want designing it.

    Bluetooth is not a reliable protocol for a board in my opinion, as you would notice if you ever use bluetooth equipment - such as audio, the signal breakup can be nasty....and occur at small ranges - much more dependant on battery levels.....

    The described behaviour will be present in any wireless system including delivery confirmation based protocols......which bluetooth is also.
     
  18. OP
    julian46

    julian46 Member

    Joined:
    Nov 14, 2016
    Messages:
    251
    Likes Received:
    101
    Location:
    Toronto, ON, Canada
    Real Name:
    Julian
    Boards Owned:
    Bamboo GT
    Wheel Setup:
    Street
    Wheel Size:
    83
    Map
    ^^^^^^^^^^^^^
    This exactly - the board should gradually stop on its own - ignoring any further commands until the communication drop out clears up

    And you are right - button / stick mashing is the worst thing you can do but common for a new rider to do - initial reaction (including grabbing a fist full of too much brake) - in my case the throttle (accelerate) command got stuck and repeated for about 3 secs - resulting in too much speed to bail on (in the mean time I did apply too much brake because it was being ignored and then it issued the full brakes instruction and I stopped dead and came off hard)

    And that incident on my first hour with the board last November led me to evolveforums.com (the one good thing to come out of this)


    Thats why we need to really get the word out:

    - some will have no issues with their board (most)
    - some may / will have issues (that can appear at any point even if you have not had a previous issue)
    - rebooting / repairing can help - otherwise hardware will need to be replaced by Evolve Service (closest to you)
    - IF you have a problem - let go of the trigger and coast it out - do not issue any further remote inputs or speed commands
     
    • Agree Agree x 1
  19. forbesmyester

    forbesmyester Member

    Joined:
    Oct 31, 2016
    Messages:
    79
    Likes Received:
    27
    Location:
    Ware, United Kingdom
    Real Name:
    Matt F
    Boards Owned:
    Bustin Mission (Gen 2)
    Wheel Setup:
    Street
    Map
    The old Gen2's braked somewhat excessively on disconnect. Some people fell off but I did not think it was a dangerous level.

    Rather than make the disconnect braking level less or better, progressive they removed it completely from the GT's
     
    • Informative Informative x 1
  20. feannorr

    feannorr Member

    Joined:
    Jan 12, 2017
    Messages:
    74
    Likes Received:
    34
    Location:
    Rathfarnham, Dublin, Ireland
    Real Name:
    James
    Boards Owned:
    Bamboo GT
    Wheel Setup:
    Street
    Map
    Bingo !!

    We have it now.....yep just get the word out.....in balance. Because like I said, I have had zero issues - however I have read all these threads.....and as a result am aware now......and will know what to do if I get a dropout.

    Sounds like a terrible first experience you had - terrifying !!!!! cuz these boards are fast !! I do not want my first fall to ever happen to be honest !!
     
    • Like Like x 3
Loading...

Share This Page