Maybe the spikes aren't our only problem (I did reground the BCMs the other day), but let's start there. The spikes probably explain the big tails, and the two noise issues might even be related.
There appear to have been intermittent, gate-level signals hitting the "BPM+BCM ROC" crate while it was digitizing. The run was nominally stopped, but a scope hung on the BCM1 ADC input showed plenty of otherwise normal activity at the sampling frequency of 500 KHz (fig 1) plus occasional fireworks that might be I/O (fig 2). Okay, maybe the ROC didn't get the memo. But I also saw intermittent large glitches which closely resemble those in Wouter and Buddhini's Ibcm1 vs time plots. (fig 3) Multi-mV transients unrelated to beam current or position are a no-no when only 0.1mV per Volt of signal corresponds to 100 ppm.
Bi-polar transients of this magnitude are not unexpected when you whack a crate with big square gate signals with nsec rise times, but Wouter's transients would correspond to O(10) msec width. Who ordered that? Did we send extra gates during the sample period? A daq expert might make some sense of it. I power-cycled the crate and all evidence of digitization stopped on the scope. It might simply come back the next run start if the problem is in the gating. We should probably do a battery test in at least one channel to sort of recertify that crate for low noise CW operations.
notes:
Read Kent's entry re funny BCMs stuff. Groan. Dinner at MacDonald's again. Katherine confirmed it with a few histograms with the large tails on the BCM distributions. Wouter showed it was in BCM1, 2, 5, 6, and indeed every BPM he looked at (they're all in the same crate). So I blamed it on the injector and went home. Just kidding. First I took the opportunity to do a sanity check of the BCM rack and found nothing untoward. (Beam was off though. Will put the scope captures in a later hclog.) Meanwhile, Wouter had brought up plots of Ibcm1 vs time which showed the large, bipolar glitches. That's usually a smoking gun for AC coupling to some sort of short square pulse. [In the background, Buddhini was assembling a Master's thesis on the topic, but I wouldn't get to read it until later.] Finally went upstairs to the ROC, looked at the signals, and power cycled the crate like Wouter and I had talked about 2 hours earlier.
probably normal 500 KHz sampling
a little scary looking, but possibly normal I/O
looks wrong. O(10) msec long transients
