Secret handshakes.

Relax in southern comfort on the east bank of the Mississippi. You're just around the corner from Beale Street and Sun Records. Watch the ducks, throw back a few and tell us what's on your mind.
User avatar
JR.
Posts: 3700
Joined: Sat May 24, 2008 7:21 pm
Contact:

Secret handshakes.

Post by JR. »

Since we don't have a digital forum I think I'll share here.

As an old analog dog I long resisted engaging with digital technology. The nice thing about troubleshooting analog circuits is that WYSIWYG. You can inspect the states of the inputs and outputs and deduce what is misbehaving. In troubleshooting digital circuitry you have another dimension or variable, where the present state of a digital circuit depends on the previous states. Not so simple. I like working in a debugging environment where you can single step through software to help identify bugs, but even with this capability it can be rough.

I have a love/hate relationship with Microchip MCUs. I love the actual microprocessors, but hate the documentation-support (long story for another time). When I encounter "quirks" (the kind word for it) in the coding, I call them "secret handshakes" because they are secrets we need to discover to make things function properly.

Just yesterday I discovered one such secret handshake that was making me crazy. The new part I am using has a sweet stereo 16b DAC built in. In the past I was using PWM to make my sine wave stimulus for my drum tuner, but the PWM was limited to roughly 9 bit amplitude resolution and not great sampling rate either. The new platform, one of Microchips DSPic MCUs has the 16b DAC using 256x up sampling, so I can push data to it at a modest rate and it digitally filters out sampling rate issues. The DAC also has a 4 sample FIFO input buffer, so I can push data to it at a 2KHz rate for a 8 kHz sample rate. sweet. But for glitch free operation, I need to detect when the buffer is empty before the 125 uSec next sample read.

No problem :roll: the DAC has a dedicated interrupt so I can call a routine to refill the buffer as needed. EXCEPT it didn't work reliably. :oops: Even more frustrating, I could get the interrupt to work a couple times, then when I added the code to refresh into the ISR (interrupt service routine) and recompiled the code It would stop working. :o

I found a mention by searching user forums with google where another coder got his interrupt to work by clearing the dac input data buffer when initializing. Presto, the interrupt started working reliably???

So, this one line of code to clear the buffer is the "secret handshake" to make this function work properly. It also explains my random behavior experience before. If the first time I tried to use the interrupt, the buffer happened to be empty. it worked properly. The next time if the buffer started out with some data in it, no workee.

Over the years I have encountered many such secret handshakes, but those are for another time.

Like I said I really like the silicon... the 16b dac is perfect for making my tuner stimulus signals... now I need to finish the FFT calculations.

JR
Cancel the "cancel culture", do not support mob hatred.
User avatar
mediatechnology
Posts: 5442
Joined: Sat Aug 11, 2007 2:34 pm
Location: Oak Cliff, Texas
Contact:

Re: Secret handshakes.

Post by mediatechnology »

John -

Sounds like there's a bug in the CPU's internal microcode. Glad you were able to find it.

I mentioned it recently in another thread but I would like to created a DSP microcode sub-forum for this place.
a) What should we call it?
b) Once we figure out a name and I remember how to create some forums can I move this post to it?

I need some PIC microcode written someday - promixe has come close to having what I need - and I might need a DSP plug-in coder soon.
I can't think of a better way to find one.

Name suggestions for the sub-forum anyone?

"JR's bits and bytes?"
"DSP and Microcontroller Projects?"
"One Infinite Loop"
User avatar
JR.
Posts: 3700
Joined: Sat May 24, 2008 7:21 pm
Contact:

Re: Secret handshakes.

Post by JR. »

Perhaps call it "things not analog" or just "Digital"

Class D power amps are analog but almost digital

I've done lots of analog tricks with digital cmos.

=========

I am almost inclined to not recommend Microchip.. While I ma hooked and not going to change horses this far down the road, if I knew then what I know now maybe I would have taken a different path, with better documentation/support.

===========
Thinking about this some more.. I don't know if it is a bug as much as a quirk of how the firmware inside the MCU works. An initial conditions thing.

Apparently the DAC buffer empty interrupt is triggered when the last sample of data gets consumed. BUT, if you never load any data into the buffer to consume, it never fires that interrupt. The DAC also has a default data output register so you can tell it what to output if the buffer is empty (normally loaded with AC zero or V/2). Several moving parts and by me clearing the data input buffer, that is the same as loading 0x0000 into it, so after it consumed that one word of data, the interrupt fires and the data buffer gets refreshed..

It kind of makes sense now, but is yet something else that they could have mentioned in the documentation. Perhaps their programming environment or initialization code incidentally loaded data so it worked for them and they never encountered it.

The part works a charm, I really like it, just wish it was less painful to learn the secret handshakes.

JR

BTW at some point there may need to be a digital hardware and digital software sub-forum... I am self taught and often feeling my way around in the dark by bumping into the virtual furniture. There may be some real experts lurking around who could chime in. .
Cancel the "cancel culture", do not support mob hatred.
User avatar
mediatechnology
Posts: 5442
Joined: Sat Aug 11, 2007 2:34 pm
Location: Oak Cliff, Texas
Contact:

Re: Secret handshakes.

Post by mediatechnology »

I've been thinking we should open a DSP/microcontroller sub-forum here.
viewtopic.php?f=6&t=614&start=7

How about calling the sub-forum "Stored Program?"
With a subtitle "Things that require Code to work."


I think for me the distinction between "I can do that" or "I don't know how to do that" or what distinguishes the two disciplines for me is whether or not a piece of code is required to pull it off.

The hybrid analog/digital tricks would seem to fit in either category particularly if it interfaces with an A/D or I/O port and requires code to work.

My gut tells me that a Class-D amp (or a switcher) is an analog thing even though it operates in the discrete time domain.
Put a DSP on the front end (or middle to switch the transistors) and then it's got a stored program and could be both.

An example of an analog thread might be a Eventide Instant Phaser. (Switched capacitor discrete time all-pass.)

An example of a "stored program thread" might be a PIC-based clock driver for BBDs that uses the analog input for CV (saw this recently) or a PIC-based SPI driver for THAT's 5171 or 5173 digital gain controllers.

Regardless of where it got posted we could move it to the appropriate thread for maximum exposure.

You should consider posting code snippets of the PIC code that works and the code that doesn't.
We get crawled by Google every 2-3 days on average and your solution would be searchable in no time flat.
User avatar
JR.
Posts: 3700
Joined: Sat May 24, 2008 7:21 pm
Contact:

Re: Secret handshakes.

Post by JR. »

mediatechnology wrote:
I've been thinking we should open a DSP/microcontroller sub-forum here.
viewtopic.php?f=6&t=614&start=7

How about calling the sub-forum "Stored Program?"
With a subtitle "Things that require Code to work."


I think for me the distinction between "I can do that" or "I don't know how to do that" or what distinguishes the two disciplines for me is whether or not a piece of code is required to pull it off.
We are at an interesting junction where high level languages get more popular and accessible.

Years ago when I was still at Peavey I speculated about a hypothetical DSP hardware platform that was the modern equivalent of the Apple II, at first mostly a lump of hardware, until software developers came up with useful applications and the rest is history.

While I haven't followed them closely there seem to be a few digital or DSP environments that are more accessible for beginners. (Arduino is one popular one).
The hybrid analog/digital tricks would seem to fit in either category particularly if it interfaces with an A/D or I/O port and requires code to work.

My gut tells me that a Class-D amp (or a switcher) is an analog thing even though it operates in the discrete time domain.
Put a DSP on the front end (or middle to switch the transistors) and then it's got a stored program and could be both.

An example of an analog thread might be a Eventide Instant Phaser. (Switched capacitor discrete time all-pass.)

An example of a "stored program thread" might be a PIC-based clock driver for BBDs that uses the analog input for CV (saw this recently) or a PIC-based SPI driver for THAT's 5171 or 5173 digital gain controllers.
I have considered publishing and selling meter boards based on my solution developed for and used in actual consoles. But I already have a day job.

I developed an automatic mixer using digitally controlled analog, that didn't make it to market because the company I was working with fell on hard times.

Publishing unfinished/unpolished code seems like an opportunity to get dragged into supporting other peoples attempts to use it.

A bunch of the thousands of lines of code in my tuner are doing obscure proprietary functions I'd rather not publish, maybe later if/when I move to a different approach. Right now I am working with FFT so this may replace my old approach.
Regardless of where it got posted we could move it to the appropriate thread for maximum exposure.
Use your judgement.. over time it will make sense, or not.
You should consider posting code snippets of the PIC code that works and the code that doesn't.
We get crawled by Google every 2-3 days on average and your solution would be searchable in no time flat.
I am kind of a dinosaur wrt coding. I prefer to write in low level assembler while the rest of the world is working in C+ or C++ . Assembler just seems more cause and effect to me. Perhaps an artifact of being an old analog dog.

I am almost tempted to write, the book that I wish I could buy explaining all the stuff I've had to figure out the hard way. But I do have a day job I should be working on right now.

As a start I may share more secret handshakes as I figure them out. At least they are interesting to me, since I know the blood and sweat involved in finding the solutions. I suspect inside a real company there will be an old gray hair that the young guys can just ask... I don't have a gray hair to ask, and find Microchip's support/documentation frustrating in that regard.

JR
Cancel the "cancel culture", do not support mob hatred.
User avatar
JR.
Posts: 3700
Joined: Sat May 24, 2008 7:21 pm
Contact:

Re: Secret handshakes.

Post by JR. »

Secret handshake #2

This is actually a work around. I am currently frustrated by the newest Microchip debugging environment, because I can not get the variable watch window feature to work for my custom variables. I can view the fixed processor registers (AKA SFRs special function registers) but my variables remain "not found" and invisible. A web search reveals that I am not alone in this problem, and I started a work ticket with Microchip support (jan 6th). So far their pattern is to wait a few weeks and then ask for more information. I expect most serious programmers will not wait 3 weeks for an answer and resolve their own questions some other way. I will let this run it's course (so far a month+ ) but I am not optimistic about a good answer from them. I suspect I may be falling between a crack where they support C+ or C++ programmers, but not assembler guys. Maybe If I wait long enough they will add assembler variable visibility. Note: It is remotely possible that I am overlooking something simple, but I do not think so. If it is something simple why hasn't support pointed me right?

OK, on to my Rube Goldberg solution. I am working on a FFT where we begin with an array of say 256 data samples, then crunch that several times to extract frequency and level information. I want to be able to see that data in intermediate results. I have done stuff like this routinely in watch windows for my earlier generation tuner, on a different debugging platform, The PIC or in this case the DSPic has flash memory that I can write to within the program. I use this mainly for keeping important system variables, when the unit is powered off. So what I figured out is I can write the array of data that I want to see to a bank of unused flash memory, and then perform a whole chip memory dump to read what is written into that memory.

I just minutes ago confirmed that it works, so now I need to refine that and get back to serious work debugging my FFT code, :D :D :D

It still isn't easy but now I can selectively perform a memory dump and inspect that snapshot of saved data.

Too much work IMO to accomplish something that should be available from within the debugging environment, but for now progress that will help me proceed.

JR
Cancel the "cancel culture", do not support mob hatred.
User avatar
mediatechnology
Posts: 5442
Joined: Sat Aug 11, 2007 2:34 pm
Location: Oak Cliff, Texas
Contact:

Re: Secret handshakes.

Post by mediatechnology »

Actually I had in mind only about 3-4 lines of code...

One example fragment would not show the instruction to pre-load the buffer; the second example would.
I wasn't thinking about a source listing for the entire tuner - just the workaround for the next poor guy that runs into this problem searching "PIC converter fails to generate interrupts." He or she isn't likely doing drum tuners but still has the problem.

The juniorhifikit thread we had recently about the summing mixer with a PIC and D/A output is an example of what would be perfect in "Stored Program."
Or your meter...
User avatar
JR.
Posts: 3700
Joined: Sat May 24, 2008 7:21 pm
Contact:

Re: Secret handshakes.

Post by JR. »

OK, this is harmless and I am always willing to share. Especially since I found this on another user forum talking about a different part.. I already informed Microchip via my open service ticket about this fix.

Here is the un-sanitized code that actually works. Life is too short for me to make code that generally only I ever look at pretty.

every thing with a ; in front of it is a text comment that is ignored by the compiler. some are orphan code that I tried and then disabled, some are notes to myself, so I can figure out what is what later.

I am not very disciplined about cleaning up my code, so you will see lots of things that didn't work just commented out
it took me a lot of tries messing with interrupt priority and all kinds of things to get it working before I stumbled on the jump start fix.

I made the magic fix blue in the code listing below...

I left in, but commented out my original fix where clearing the data register made it work. An old programming habit when coding by trial and error, so you can undo experiments and restore previous working code.

I changed that register clear (that worked) to loading actual data 0x0001 into it, to confirm it wasn't clearing it that made it work, but setting it to any real value. You can't literally write data into the register, you have to write it into a working register (like w0) first, then transfer than woking register into the data buffer.

There is lots of other stuff there associated with initializing the DAC related to setting up the output timing and DAC configurations etc. I call that routine once when I first start up the software.

To be specific this is my fix for MICROCHIP DSPIC p33FJ128GP802 DAC buffer empty interrupt startup problem... simply stated pushing one word of data into the buffer jump starts the interrupt working, when that one word of data is consumed.

enjoy

JR 8-)

====code----
_init_DAC:
;pins 20, 21 DAC1R
clr w0
;pll post n2 div/8
bset w0, #7 ;div 8
bset w0, #6 ;div 8
;pll pre n1 4
bset w0, #1 ;div 4
mov w0, CLKDIV

;M div/71 50
clr w0
bset w0, #5 ;32
bset w0, #4 ;16
bset w0, #0 ;69+2=71
mov w0, PLLFBD

clr w0
bset w0, #8 ;div/64
;bset w0, #9 ; /4
bset w0, #10 ;div /1
mov w0, ACLKCON
; bset ACLKCON, #7 ;1 = Primary Oscillator is the Clock Source ASRCSEL
mov #0x0000, w0
clr TMR1
mov w0, DAC1DFLT

clr w0
; bset w0, #1 ; div by 3 on
bset w0, #8 ; signed data
bset w0, #15 ;on
; bset w0, #3 ;div/10
bset w0, #1 ;div/10
mov w0, DAC1CON

clr w0
bset w0, #7 ;ROEN right on
bset w0, #2 ;;int if buffer empty
mov w0, DAC1STAT


mov #0x0800, w1
mov w1, NotePeriod_L
; ASR, w1, w1
; mov #NotePeriod_L , w0
; add w0, #4, w0
; mov w1, [w0]
; mov w1, NotePeriod_L+#2
; clr NotePeriod_L
call load_new_periods
mov #0x0001, w0
mov w0, DAC1RDAT
; clr DAC1RDAT

bset IPC19, #9
bset IPC19, #10
; bset IPC19, #8
bclr IFS4, #14
bset IEC4, #14
; bclr IFS4, #15
; bset IEC4, #15

bclr SR, #5
bclr SR, #6
bclr SR, #7
bclr IPC19, #12
bclr IPC19, #13
bclr IPC19, #14

return
Cancel the "cancel culture", do not support mob hatred.
User avatar
JR.
Posts: 3700
Joined: Sat May 24, 2008 7:21 pm
Contact:

Re: Secret handshakes.

Post by JR. »

To follow up on SS#2 Microchip support finally came through and advised me of the work around for the bug in the IDE environment, so more than a month later, I can finally see my variables. Better late than not at all. :D
=====
Secret Handshake #3 ... Not sure if this qualifies as a full blown secret handshake but studying the FFT code (that I did not write) that I am using inside my next generation, I found some clever assembly language coding. Since I've never seen a comprehensive description of how all these assembler instructions work I have to learn this stuff by inspection.

There is a hardware DO loop built into the MCU firmware that support Do loops without extra processor or code overhead. A software loop can use a couple extra instructions per loop pass to increment a loop counter and then compare it to a fixed number to branch out of the loop.

The hardware loop handles this in hardware (firmware?) so there is a SFR (dedicated special function register for the loop count that gets decremented one count for each loop through. There are also SFRs for the loop start address and the loop end result. Further the loop count can be loaded from another working register allowing dynamic behavior in loop execution based on some variable's status.

Further, and this is the secret handshake that I observed while debugging some FFT code, is that if you step into a DO loop, with the loop count variable set to 0, and the loop end address does not get changed or updated. This way DO loops can be nested within other DO loops can execute dynamically depending on the variable status when it passes through the top of inner loops. When an inner loop finishes it does not automatically return to the outer loop, but has to reenter it from the top, but keeping track of the outer loop counters allows for some complex calculations with relatively simple and efficient code.

FFT calculations are one of those software problems where people invest a lot of though looking for efficient implementations because we can be talking about thousands of individual calculations.MCU even set up dual memory partitions so you can manipulate two banks of data with one instruction. To execute the FFT butterfly calculations first you reorder the serial data into a bit reversed order, so then the nested do loops can give a good result.

If course this is an overly simplistic explanation of just one of the sundry radix-2 FFT algorithm. The secret handshake IMO is how the DO loop operates differently depending on the value of loop count variable. I would not be surprised if this was no accident, but done in purpose to help make an efficient FFT algorithm. Even the bit reverse addressing scheme is a part of the DSP hardware instruction set modulo addressing capability to facilitate efficient DSP code.

Sorry if this is too obscure... Interesting to me... while I'm in the deep end of the pool, treading water.

I just got the FFT running and this looks promising for my next gen tuner..

I really have to marvel at modern technology. I can calculate a 256 point FFT on a few dollars worth of silicon in about a second... I can do the math faster than the drums will stabilize from the stimulus.

JR
Cancel the "cancel culture", do not support mob hatred.
User avatar
JR.
Posts: 3700
Joined: Sat May 24, 2008 7:21 pm
Contact:

Re: Secret handshakes.

Post by JR. »

Just got bit by secret handshake #1 again.

To refresh our collective memory SHS #1 involves the interrupt for the 4 sample deep DAC buffer being empty, not firing if we haven't loaded the data buffer first so it can "become" empty.

I was working on some later code that altered the way I use the DAC buffer. It has a 4 deep buffer that is nice for more efficiently calculating and loading multiple sequential samples of data, but for some of my algorithms I need to be able to trigger other events on a per sample basis, So I shift the interrupt trigger from "fire when buffer is empty", to "fire when buffer is not full" For updates with each individual sample for those selected modes of operation.

I discovered that sometimes when changing between modes of updating the buffer, the DAC would hang up and stop sending interrupts again...

So this secret handshake is apparently useful to know. Now I just push one sample of data into the buffer after changing the status of the interrupt trigger, and it works reliably.

JR
Cancel the "cancel culture", do not support mob hatred.
Post Reply