Using stdio via usb

[BjornTheProgrammer]

I don't know if anyone else is having this issue. When I attempt to run the code using the default settings in CMakeLists.txt for stdio, there are no issues, the light blinks properly. However, when I attempt to use USB instead of UART, the light stays a solid green and no output is given over serial.

I also attempted to git reset --hard and run the code again, and only change the

pico_enable_stdio_uart(piccolo_os_demo 1)
pico_enable_stdio_usb(piccolo_os_demo 0)

to

pico_enable_stdio_uart(piccolo_os_demo 0)
pico_enable_stdio_usb(piccolo_os_demo 1)

I don't have any way of checking the data coming from the UART stdio, however, I've compiled and gotten output over USB from other pico projects, and haven't had issues.

[BjornTheProgrammer]

I've managed to track down the issue to the context_switch.s assembly function __piccolo_task_init_stack, I am still attempting to debug this issue, and will post the results of the findings later.

[garyexplains]

I think the reason I couldn't do full context switching based on a timer/interrupt is because of some odd interaction with the SDK. I think you are seeing a similar thing. I got full context switching working on other Cortex-M0+ boards, just not the Pico. The only difference I am guessing is the Pico SDK. I wonder if the USB code uses interrupts?

…

On Fri, Jul 1, 2022, 05:43 Bjorn ***@***.***> wrote: I've managed to track down the issue to the context_switch.s assembly function __piccolo_task_init_stack, I am still attempting to debug this issue, and will post the results of the findings later. — Reply to this email directly, view it on GitHub <#3 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AJUW52GCKBDKVT7RUP5UQW3VRZLODANCNFSM52FDIYNA> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[BjornTheProgrammer]

Sounds reasonable, I'm going to try examine the difference between the USB stdio printf behavior and the UART stdio printf, to see if I find anything.

[BjornTheProgrammer]

I have an update. I was rereading the getting started guide for the raspberry pi pico, specifically the debugging section on page 22, and found the following...

When using SWD for debugging you need to use a UART based serial connection (see Chapter 4) as the USB stack will be paused when the RP2040 cores are stopped during debugging, which will cause any attached USB devices to disconnect. You cannot use a USB CDC serial connection during debugging.

Clearly, the only way to do context switching with the added functionality of serial USB output would be to determine somehow where the USB stack is and preserve it.

It is beyond my wheelhouse to even begin to do this, but if anyone has deeper insight into assembly language and how stacks are allocated via USB serial connections, I'd be willing to help find a solution.

[KStandiford]

@BjornTheProgrammer - The Pico SDK needs to run in privileged thread mode. __piccolo_task_init_stack sets the CONTROL register to #3. Change this to #2 to enable privileged access for thread mode. USB then works on STDIO. I suspect that a lot of other parts of the SDK work better as well. It might even be worth having another go at pre-emption as well!

[garyexplains]

Thanks @KStandiford that seems to have been the problem. I have tested and merged your PR and USB output now works. Thanks again.

[garyexplains]

FYI @KStandiford I had another go at pre-emption and now it works via SysTick! 😁 So thanks again for your help and insight.

[KStandiford]

Thanks @garyexplains! I got it working as well with systick. I DID NOT use PENDSV however. The Pico SDK uses spinlocks to protect critical code between cores, and between regular code and irq handlers. The first thing they do is disable interrupts. However PENDSV cannot be disabled, and Systick is not covered by the code either, so preemption using PENDSV can break the spinlocks. Instead I used a user IRQ and let its interrupt handler do the context switch. This IRQ WILL be disabled by the spinlocks. (I also lowered the irq priorities of SVC, Systick and the user IRQ. I did not want to preempt other IRQs.) This method also takes a little jiggling to handle race conditions between the task yielding and Systick going off. BTW @BjornTheProgrammer, there are other bugs in the USB serial code that confused me for awhile, but serial UART works fine. USB stdio seems to loose data even without Piccolo.  I think I can get multi-core scheduling to work, but I will need to go on and create a task struct to clean things up a bit. (Which can lead to dynamic allocation of tasks as well. Malloc and company ARE multicore safe.) There are a few other things I will try to add and clean up.  Thanks again for doing this. No additional commas required, it is a great teaching tool: minimalist for simplicity and lesson focus and explained about four different ways. I learned! On Saturday, August 13, 2022 at 04:59:11 AM PDT, Gary Explains ***@***.***> wrote: FYI @KStandiford I had another go at pre-emption and now it works via SysTick! 😁 So thanks again for your help and insight. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: ***@***.***>

[BjornTheProgrammer]

@KStandiford would you be willing to share your code for getting preemptive context switching with the raspberry pi pico? Would be forever grateful.

[garyexplains]

@BjornTheProgrammer Here is a GIST with the changes you need to make to use systick, however it doesn't deal with the issue of the spinlocks that @KStandiford mentions https://gist.github.com/garyexplains/1bc33eb44716ec5ff6f0ce43eb44089d

[KStandiford]

@garyexplains and @BjornTheProgrammer
I will see if I can retro-fit into the V1.0 framework and and test it in the next day or so. I will need to write a short document that explains everything as well.

I need to retrofit, because I got a bit crazy and added a few things:

• Multi-core operation with the scheduler on both cores running any available task
• Dynamically allocated task structures and stacks (using malloc())
• Task exit and deletion (using free())
• Task blocking without execution (blocked tasks do NOT run)
• Task signalling with optional blocking and timeouts on sending or receiving
• Sleep blocks rather than spinning until the timer expires
• An idle task which sleeps for awhile to reduce power consumption
• And (sorry) various PEDANTIC changes

Multi-core wasn't that hard. Task deletion was &^($% hard! (My wife started laughing every time I announced that I had finally fixed it.) Epiphany is such a mixed emotion - I always am gobsmacked by how stupid I have been.

I am in the clean-up and documentation phase (yikes there is a lot of debug garbage), but I plan to post it next week. I was thinking of Version 1.5 or Piccolo_Plus? Gary, you started this and I don't want to be presumptuous, so I'm interested in your thoughts before I put it out there. I don't want to have this look like I'm trying to steal any thunder of one-up anyone. I will be sure to preserve original documentation and copyrights.

[garyexplains]

Wow, that is some great work. I think the best approach is for us to keep V1 as it is (for teaching) and expand the documentation to explain how pre-emptive can be added, for those interested.

But we should also start a V2 repo with all your improvements and then add other things like locks, queues, etc.

What do you think?

[BjornTheProgrammer]

I think it would be a fantastic idea to open a v2 repository, and keep this as v1. Honestly @KStandiford and @garyexplains thank you so much for doing this. I've been reading documentation for literally months on how to get preemptive context switching working, and learning asm, along with some friends, it means a lot. Also I can test it on my raspberry pi pico w and raspberry pi pico.

[KStandiford]

@garyexplains and @BjornTheProgrammer
OK, here are the changes I made and a document about the reasoning.[(https://gist.github.com/KStandiford/785bbea1e542ee4032a0ed28d38ed57b)] I have also uploaded a preemption branch to my piccolo_os forked repository, so it can be seen.
I also love v1 as a teaching aid, but I think there are a couple of minor pitfalls that could be fixed to save students (like me!) some grief. My suggestions are in the pedant branch of the same repository and are also explained here: https://gist.github.com/KStandiford/2e7485e226ffe8f3475d526c57dffccc
I also like the idea of v2. But I think we have to be careful to set the bounds. After all, there is a FREE RTOS out there. I'm a minimalist and I try to live by the KISS principle (Keep It Simple, Stupid!), because the last 'S' is for me. The more feature rich a v2 gets, the harder it will be to learn and the more likely it will be to break itself or the SDK in obscure ways due to it's simplistic nature. But I did the stuff because I wanted to see if I could, I wanted to learn about it, and I wanted to use it in my other projects. And thanks for the complements.

[BjornTheProgrammer]

I just believe it would be a shame to see all this code buried under a closed issue thread, without anyone else being able to easily discover a working preemptive context switching mechanism designed for the raspberry pi pico with the pico sdk.

[KStandiford]

@garyexplains and @BjornTheProgrammer
While thoroughly confused by my own bug in "2.0" I realized just how fragile 1.0 with preemption is. Preemption in the malloc(), free() world will hang (Think C++!), as well as anything else using mutexes inside the PDK. So I found how to really integrate v1 with the SDK so that ALL the fancy SDK features like mutexes, semaphores, queues, etc should work. Parts of the SDK that need to block for these things will call piccolo_yield() nicely. I posted the details on the previous GIST and branch of my forked v1 repo.

I do think we should post this, though NOT as 2.0. Call it 1.5, or 1.0+ or v1_with_preemption? (And please include the pedant stuff! I wasted another 2 hours because I forgot again that it runs without preemption on task creation!) I have a version I would like to propose for a 2.0 starting point, but need to get the doxygen part of the work done so it will document itself nicely. I need most of another week.

[BjornTheProgrammer]

@KStandiford Amazing strides, tell me if there is anything I can do to help out.

[garyexplains]

First let me echo the compliments of @BjornTheProgrammer, great stuff @KStandiford. Many thanks for all your efforts and insights.

I have been reading over your stuff in the gists and it looks excellent. One question, does __piccolo_systick_config(PICCOLO_OS_TIME_SLICE) need to be called every time in the while loop in piccolo_start()?

Besides that, I think the code and the documentation you have provided should undoubtedly be made more visible and public.

I was thinking of backporting your "box of commas" stuff to v1.0 as it is undoubtedly the right way to go. However, that would mean re-writing all the documentation for v1.0 and it would also mean the codebase is out of sync with the video.

So, here are my current thoughts:

• Create v1.1 which includes all of your changes for preemption including the "box of commas" stuff and the other preemption stuff plus the locks for malloc etc.
• Create a v2.0 from v1.1 that can be used to implement more stuff like queues, semaphores etc.
• I will make a video on Gary Explains about v1.1
• I will make a series of videos on Gary Explains about v2.0 as it is a good teaching aid to explain the concepts of different things like semaphores and how they can be implemented.

To help limit feature bloat for v2.0 I commend that

• a) Piccolo OS remains strictly for the Raspberry Pi Pico. This has basically already happened as all the stuff we are talking about is related to the Pico SDK and how to work with it to make Piccolo OS.
• b) Aim to keep it simple. So for example we could never support dynamic task creation/destruction, or anything other than a round-robin scheduler.
• c) If something already exists in the Pico SDK, then we don't need to re-invent it for Piccolo OS. I am thinking about things like networking etc. As long as Piccolo OS plays happily with the SDK then everything should be good.

Finally, I think v1.0, v1.1, and v2.0 should be separate repos. I know that we could use branches, labels, etc. But for a "student" to see clearly each thing then I think separate repositories are best. They can look at v1.0 and then v1.1 and then move to v.2. We don't want the v1.0 stuff buried under branches and tags etc. The v2.0 repo can have more sophisticated release management, if we want.

Thoughts? Comments?

[KStandiford]

@garyexplains and @BjornTheProgrammer: Thanks guys! (Now my hat doesn't fit...)

First, I think calling __piccolo_systick_config(PICCOLO_OS_TIME_SLICE) every time in the while loop in piccolo_start() is important. One reason is the concept of "fairness" behind the first paragraph in my GIST. Resetting the timer before every task runs makes sure that the task gets the full time slice microseconds of computing. Without resetting the timer each time, preemption comes at completely random times from the task point of view. (Yeah, it sort of does anyway but...) The timer may even expire while the scheduler is running, in which case the pending exception will be serviced as soon as the next task starts, and the task will miss an entire turn! This last case is why __piccolo_systick_config() is so careful about disabling the timer and then clearing any pending timer interrupt. (See the section on race conditions in the GIST). Admittedly, this does increase the system response time given multiple tasks that don't yield fast enough. If attempting to set a "limit" on the system latency is the objective, then the reloading of the counter (writes to rvr and cvr registers) could be moved to an initialization routine and __piccolo_systick_config() probably gets a new name.

Separate repos is absolutely what to do. I only created branches to communicate and share with you guys. I like the versioning you propose. I also understand and applaud the KISS approach for teaching tools. For teaching, I think your scope rules for v2.0 are perfect. In fact, given the rich set of tools in the SDK that should work in v1.1 (mutexes, semaphores, queues, spinlocks, etc.), I think v1.1 is v2.0. Not to say that demonstration implementation of such things is not educational, of course. The only thing I can think of to add to v1.1 that fits your guidelines is multi-core, which I would consider backfitting as a proposal.

My goals, however, are a bit different. I like learning. My preferred method is to find a project where there is a need that interests me, and to take a deep dive to synthesize a system overview. I then try to create a minimalist, professional grade tool with sufficient features that others might find it useful, integrate and optimize it as much as I can learn to do, and then over document it. (See my Fast-LCD-I2C repo). My version is already way beyond the scope limits you set for v2.0, and is too complex for a teaching tool.

So what I propose is that I publish separately and (with your permission, I hope) call it Piccolo-OS-Pro or Plus, or something like that. I would also be happy to help on v2.0. What I would like to ask of you, since it is hard to get noticed out there, is as many free plugs as you could stand to give out.

Thanks for reading to the end!

[garyexplains]

OK folks, the piccolo_v1.1 repo is live and kicking: https://github.com/garyexplains/piccolo_os_v1.1

Keith, I was 90% way done with the v1.1 repo when I saw your last message. I hope you are happy with how it looks. I think I have added all the right copyright notices and given full credit where necessary. If I have missed anything then please let me know.

As for your own fork of Piccolo OS, please do go ahead and publish and continue to develop it as you see fit. Let me know the URL and I will include pointers to it in the documentation and in any future videos I do about Piccolo OS.

Talking of videos, my aim now is to create an updated video covering all this new stuff.

I will let the dust settle on this and then think about v2.0, as you say Keith maybe the SDK has enough already.

[BjornTheProgrammer]

It's still a work in progress and semi-buggy, but I wanted to show you guys what I've been working on for the past few weeks: making using the Raspberry Pi Pico painless. I noticed it was a real pain to get the Pico working on anything other than the Raspberry Pi (Windows, Mac, Arch, etc.), so I decided to create an interface that can be used from anywhere, and by multiple people at once.

That's when I created Pico-Online, it has an editor, terminal, and file manager which allows someone to build code to the Pico at any time using a Raspberry Pi 4. I currently set up a website for my Pico W at home. You can see the site at berdos.org. I currently have four people using the web app for another project to test and deploy code. You can check out the code here. The goal is for this to eventually be a one-command install, just type ./install.sh and the app installs and starts up, and people can start running code on their pico.

The nice thing is that the project that the other four people are using Pico-Online for requires a lot of hardware, and it gets expensive the more you purchase one at a time for them and having one set of hardware accessable from anywhere to test code on is much cheaper and hassle free. I imagine that this setup is good for any remote Picos people have (maybe outside, sensors, etc.), that people would like to be able to update at anytime.

[BjornTheProgrammer]

Additionally, Pico-Online is meant to sync up, so when multiple people use Pico-Online (you can open another window to test this), and run a command like pull or build, you will see the output in the terminal in both windows.

p.s.
Please don't mind the fact that the code doesn't build on any of the branches, it's our source code (not Pico-Online) that is the issue.

[garyexplains]

Wow, that is some great work. I think the best approach is for us to keep V1 as it is (for teaching) and expand the documentation to explain how pre-emptive can be added, for those interested. But we should also start a V2 repo with all your improvements and then add other things like locks, queues, etc. What do you think?

[BjornTheProgrammer]

@garyexplains Standiford has already made a repository with the changes, you can view it under Piccolo_OS_Plus.