5 min later, after it recovered itself from the alarm, the tgt-ioc, iochc4 (on the remote reboot panel), flatlined. Rebooted it with Chris K. on the phone (he asked for a heads-up when it happens to monitor the JT's behavior on recovery). Eveything recovered fine except for the alarm handler (it had all white signals). Killed it, and restarted it, still all white, but now all medm panels are white (strip-tools are updating fine, which means that the ioc is alive and there is some connection between the hall and CH). I checked for dead local medm processes on gzerol2, found two and killed them, still white signals in medm.
~5:00 called Chris K., he suggested to login into the machine in the hall and starting the medm from there (he has it running fine from home). Tried it, still white signals on gzerol2's screen. Chris suggested rebooting the ioc again (it happened ~5:10am). All recovered fine this time. Restarted the alh and with Chris changed some limits as follows:
1. deactivated the alarms on the refrigerator and the separator 2. changed the low limits on flow for LN2 and LHe from nominal to 10 and 20 (LN2), and 10 and 15 (LHe) respectively 3. He pressure in the magnet dewar (PI9112) HI and HIHI to 1350 and 1370.
The Windows machine in the hall seems to have the automatic updates ON. It finished updating overnight and wants to restart. It may be sensible to deactivate this feature during daq.
5:50 before submitting this entry, I checked the monitors and noticed that the coil temperatures were all flatlined at 0, so I rebooted the tgt-ioc again. All recovered fine. Re-set alarms as agreed with Chris for this night.
All this time there were about 580nA of cw beam in the hall (at about the
nominal positions in 3A and 3B from 10.31.08).
Figure 1
