Analyzer/Comparisons

Setting up Fortran Analyzer (ENGINE) and Replay

Get copy of Analyzer and Replay directories

• The ENGINE code has been put in a github repository.
  • To checkout and contribute using github:
    • Create an account on github.com and following the directions for storing your public ssh key on the site.
      • NOTE: The github instructions ask you to install a program called "xclip" to help upload your public key. This is unnecessary - you can just "click and paste".
    • Once you have signed into github then go to https://github.com/JeffersonLab/engine.
      • Select Watch to be notified of changes to engine repo and the click on the Fork button to create your own Github copy of the repo.
    • On the computer that you want to run the code
      • Create an "analysis" directory for doing the ENGINE replay.
      • In the "analysis" directory type: git clone git@github.com:Githib-Username/engine.git where Githib-Username is your github username.
      • This creates the subdirectory engine.
      • Now "origin" remote is to your personal Github repository. After cloning you will be in the "master" branch. You need to use the "mkj" branch.
    • Always work in the "mkj" branch or in a new branch that uses "mkj" branch as starting point. Make your local changes and commit them to your local copy. You can update the github repository by command: git push origin mkj . If you want to have these changes incorporated into the JeffersonLab repository then go to your github account Githib-Username/engine and go to branch "mkj". Click on the pull request button. This will notify the person monitoring the JeffersonLab repository and your changes can be pulled in.
    • To have access to the changes in the JeffersonLab GitHub repository, in your newly created engine directory
      • One time need to execute the command: git remote add upstream --track mkj git@github.com:JeffersonLab/engine
      • The command git fetch upstream/mkj will fetch changes
      • The command git merge upstream/mkj will merge any changes fetched into your working files
• Use the ENGINE code:
  • go to engine subdirectory
  • git checkout mkj
  • The "mkj" branch has been updated to work on the JLab CentOS 6.2 machines (ifarm). It will work on the CentOS 5.2 machines (ifarml64) but these are being phased out at JLab. The branch also has modifications to make comparisions to HCANA code. The version of the ENGINE is based on the inclusive data analysis.
  • Type "setup cernlib/2005" to set CERN=/site/cernlib/x86_64_rhel6 , CERN_ROOT=/site/cernlib/x86_64_rhel6/2005, CERN_LEVEL=2005 on CentOS 6 machine.
  • Type "make" in engine subdirectory. This will take some time with lots of WARNINGs and should create the executable Linux/bin/engine-replay ( Linux is a subdirectory of the "analysis" directory).

Get the ENGINE replay files directory

• The ENGINE replay files have been put in a github repository.
  • To checkout and contribute using github ( assume you have account):
    • Once you have signed into github then go to https://github.com/JeffersonLab/replay.
      • Select Watch to be notified of changes to the repo and the click on the Fork button to create your own Github copy of the repo.
    • On the computer that you want to do the replay
      • In the "analysis" directory type: git clone git@github.com:Githib-Username/replay.git where Githib-Username is your github username.
      • This creates the subdirectory replay.
      • Now "origin" remote is to your personal Github repository. After cloning you will be in the "master" branch.
    • Make your local changes and commit them to your local copy. You can update the github repository by command: git push origin master . If you want to have these changes incorporated into the JeffersonLab repository then go to your github account Githib-Username/replay. Click on the pull request button. This will notify the person monitoring the JeffersonLab repository and your changes can be pulled in.
    • To have access to the changes in the JeffersonLab GitHub repository, in your newly created engine directory
      • One time need to execute the command: git remote add upstream git@github.com:JeffersonLab/replay
      • The command git fetch upstream will fetch changes
      • The command git merge upstream will merge any changes fetched into your working files

Updates to the "mkj" branch of ENGINE

• Add SOS Drift chamber variables to h_dc_ntuple .
  • Move call h_dc_ntuple_keep to g_keep_results so that it is called for all event types
  • Update h_dc_ntuple_keep
    • put SOS DC variables into ntuple
    • have HMS variables start with hdc
    • hardwire to only write out SOS events (evtype =2)
  • Update h_dc_ntuple_open
    • have HMS variables start with hdc
    • add SOS DC variables under SDCINFO common block
    • add iquest(10)=65000 to allow larger ntuple file size
  • In h_keep_results : remove call h_dc_ntuple_keep
  • In gen_pawspace.cmn increase size of G_sizeHBOOK to 8000000
  • Update h_dc_ntuple.cmn
    • have HMS variables start with hdc
    • add SOS DC variables under SDCINFO common block
• In HTRACKING/h_targ_trans.f : Set hut(5) = 0.0 to match HCANA until beam is put into HCANA. Skip the COSY sums part that is explicitly for Jan05 analysis.
• In HTRACKING/h_sp_multiwire.f : Put in an "if" statement to skip the reordering of the space point hits to match what is done in HCANA. Has no effect on results.
• Add parameter hdc_fix_propcorr. When a hit gets put into a stub, the distance of the hit from the discriminator can then be estimated. In the engine, a correction to the drift time (and thus drift distance) is applied. However, if that hit ends up in another stub, the correction will get applied again, resulting in a over correction. Setting this flag to 1 will give that hit a different correction for each stub that it is in.
• Add parameter hdc_fix_lr. Historically, in the ENGINE, if a hit is used in multiple space points/stubs, the left/right assignment for that hit, which is later used in track fitting, is that assigned for the last stub encountered with that hit. Properly, the left right assignment should be allowed to be different in different space points. If this parameter is set to zero, then the original ENGINE behavior is used. For new analyses, it should be set to one.
• Modified "gstart" and "gstop" parameters to refer to the CODA event ID number. Makes it easier to compare to HCANA when HCANA is replayed with analyzer->SetCountMode( 2 ) so that it is counting based on the CODA event ID number.
• Created column-wise ntuple with the name defined by parameter h_dc_ntuple_file .
  • In ENGINE added code h_dc_ntuple_clear.f, h_dc_ntuple_init.f, h_dc_ntuple_keep.f, h_dc_ntuple_register.f, h_dc_ntuple_shutdown.f,h_dc_ntuple_close.f,h_dc_ntuple_open.f, h_dc_ntuple_change.f .
  • In INCLUDE added h_dc_ntuple.cmn, h_dc_ntuple.dte
  • In ENGINE modified Makefile.Unix, g_ntuple_init.f, g_ntuple_shutdown.f, h_keep_results.f, h_register_variables.f
  • To add varaibles to the ntuple:
    • In INCLUDE/h_dc_ntuple.cmn, one defines the common blocks and variables that will be included in the ntuple.
    • In ENGINE/h_dc_ntuple_open.f, one makes the call HBNAME for each common block.
    • In ENGINE/h_dc_ntuple_keep.f, fill the variables.
    • Presently there are variables for HMS tracking.

Getting data from the MSS

• Most of examples are for run 52949 which might already be on the cache disk since people are using the file.
• For detailed instructions see new jcache commands)
• An example is : jcache get /mss/hallc/daq04/raw/daq04_52949.log.0

Replaying a run with the ENGINE

• Need to have run number 52949 from daq04 experiments on the cache.
• cd to the replay subdirectory in your "analysis" directory
• Type " replay_inter grun=52949 gstop=10000 gtrig2=1 gtrig3=0 gtrig4=1"
  • This will replay up to event number =10000 and analyze only HMS and COIN event types and create the hbook file ( paw/52949.hbook) and a paw kumac which has the aliases for the histograms ( paw/52949.kumac ).
  • gtrig2=1 (or 0) means analyze (or do not analyze) HMS trigger (event type 1)
  • gtrig3=1 (or 0) means analyze (or do not analyze) SOS trigger (event type 2)
  • gtrig4=1 (or 0) means analyze (or do not analyze) COIN trigger (event type 3)
• To make comparison to HCANA replay, convert the hbook file to root file using h2root.

Replaying a run with the HCANA

• Need to have run number 52949 from daq04 experiments on the cache.
• Assumes that one has checked out the hcana repository and has compile the hcana code.
• Mark Jones created a separate hcana replay directory from the normal hcana/examples subdirectory.
• The hcana replay files have been put in a github repository. To get the code follow the same directions used to get hcana, but replace "hcana" with "hcana_replay". When setting up the "upstream" remote, leave off the "--track develop" argument.
• In the "analysis" directory :
  • extract the hcana replay files: "git clone git@github.com:GitHub-Username/hcana_replay.git". This creates the subdirectory replay.
  • Create symbolic link: " ln -sf ../replay/DBASE ."
  • Create symbolic link: " ln -sf ../replay/DATFILES ."
  • Create symbolic link: " ln -sf ../replay/MAPS ."
  • Create symbolic link: " ln -sf ../replay/PARAM ."
  • For raster decoding, we had to modify the ENGINE type map files by adding a new detector ID and a plane ID.
    • For replaying jan05 data, a new map file "july04_hcana.map" and new database file "jan05.database" have been created and the hcana macros call "jan05.database"
    • In general to replay with other "replay" directories
      • Use the script "/Macros/mapedit_rasterbcm.pl" to convert ENGINE type map files to this modified type.
      • So as to not interfere in the engine replay directory, create a new map as e.g. "cp MAPS/july04.map july04_hcana.map"
      • Copy the database file that you use for the engine replay and edit it to use the new map file.
• Replay a run
  • cd to the hcana_replay.
  • If you have compile the hcana code then you should have the environment variable HCANALYZER define, then make symbolic link : " ln -sf $HCANALYZER/hcana hcana"
  • Type "hcana" and get the prompt:"analyzer [0]"
  • To analyze, Type ".x replay_both.C(52949,1,100000)" at the prompt. Creates the file replay_both_52949.root in the Rootfiles directory.

Compare HCANA to ENGINE for HMS hodoscope and shower

• The root macros assume that the ENGINE root file is in the paw directory that is a subdirectory.
  • Type" ln -sf ../replay/paw paw" to create this subdirectory.
  • Then one can do : "h2root paw/52949.hbook paw/52949.root".
  • Type "root" in the hcana_replay directory
• All plots show the ENGINE (red line) plotted over the HCANA (blue line) in one column. If perfect then just see red line. In other column is the difference between the ENGINE and the HCANA plot. If perfect then zero counts in the histogram.
• For hodoscope ADC hit map:".x Macros/plot_hodo_adc.C(52949)".
  • Scint 1 adc hit map,Scint 2 adc hit map
• For hodoscope TDC hit map:".x Macros/plot_hodo_tdc.C(52949)".
  • Scint 1 tdc hit map,Scint 2 tdc hit map
• For hodoscope focal plane time:".x Macros/plot_fptime.C(52949)"
  • Run 52949 plot of focal plane times with cut of track in DC. Perfect match.
• For hodoscope comparing focal plane times and start time on per event basis: ".x Macros/comp_hms_hodo_engine_hcana.C("Rootfiles/replay_both_52949.root","paw/hms_dc_52949.root")"
  • plot of focal plane times for all HMS triggers.
  • plot of difference between ENGINE and HCANA focal plane times. All within 1x10-5.
  • plots of start time (top plot) and difference (bottom) between ENGINE and HCANA start times. All within 1x10-5.
• For calorimeter ADC hit map:".x Macros/plot_shower_adc.C(52949)".
  • Adc hit map
• For calorimeter individial ADCs :".x Macros/plot_shower_ind_adc.C(52949)".
  • Col A Neg 1-6, Col A Pos 1-6, Col A Neg 7-13, Col A Pos 7-13.
  • Col B Neg 1-6, Col B Pos 1-6, Col B Neg 7-13, Col B Pos 7-13.
  • need to modify the macro to look at columns C and D.

Compare HCANA to ENGINE for HMS drift chambers

Replay the data

• Replay the data with HCANA
  • Go to your hcana_replay directory and check that run 52949 is on the cache disk.
  • If you have compile the hcana code then you should have the environment variable HCANALYZER define, then make symbolic link : " ln -sf $HCANALYZER/hcana hcana"
  • Type "hcana" and get the prompt:"analyzer [0]"
  • type " .x replay_both.C(52949,1,100000)" to replay up to CODA event ID number =100,000
• Replay the data with ENGINE:
  • cd to the replay subdirectory in your "analysis" directory
  • Type " replay_inter grun=52949 gstop=100000" to replay up to CODA event ID number =100,000
  • Should create ntuple file paw/hms_dc_52949.rzdat . If not then check that h_dc_ntuple variable is set in the PARM_FILES/REPLAY_FARM_COIN.PARM
  • To make comparison to HCANA replay, convert the hbook file to root file using h2root.

Macro to compare

• Compare HCANA and ENGINE:
  • Go to your hcana_replay directory. Assume that you have a softlink to the paw directory to have " ln -sf $SOME_DIR/analysis/replay/paw paw"
  • Type "root"
    • .L Macros/analyze_hcana_tree.C
    • .L Macros/analyze_engine_tree.C
    • .x Macros/comp_hms_engine_hcana.C("Rootfiles/replay_both_52949.root","paw/hms_dc_52949.root")
    • You will get the comparison plots shown below.
  • You can modify comp_hms_engine_hcana.C to print out track information. For example:

hcana event number = 1038 event type = 1
Number of Total hits  = 22
Number of Tracks  = 2
Track focal plane  x (cm)    y (cm)    dx/dz     dy/dz
   1              -20.3977    7.8901  -0.029469  0.012129
   2              -20.3970    7.7412  -0.029492  0.016835
Print track focal plane is not sorted by chi-squared
Track target        y (cm)   dx/dz     dy/dz       delta      chi2
  1                -0.778     0.023   -0.01325    -4.86606   67.85777
  2                 1.523     0.021   -0.02869    -4.88804   256.37274
Print track target info sorted by chi-squared
Engine event number = 1038 event type = 1
Number of Tracks  = 2
Track focal plane  x (cm)    y (cm)    dx/dz     dy/dz
   1              -20.4031    7.7859  -0.029295  0.015258
   2              -20.4029    7.7468  -0.029301  0.016664
Print track focal plane is not sorted by chi-squared
Track target        y (cm)   dx/dz     dy/dz       delta      chi2
  1                 1.433     0.021   -0.02821    -4.90070   248.31000
  2                 0.719     0.021   -0.02392    -4.89441   517.14258
Print track target info sorted by chi-squared
Hit return to see next event

Comparison plots

• Focal plane position and angles for events with number of tracks greater than or equal to one. The plots are 6 histograms: the top plots are X and Y ( or Xp amd Yp (mr)) with results from HCANA (black) plotted are top of ENGINE results (red), the middle plots are the difference between the top histograms, and the bottom plots are for each event the difference between the HCANA and the ENGINE with the scale of +\- 0.0001 . For the bottom plots of the angles the difference is in radians not mr.
  • For the 1st track Positions, Angles
  • For the 2nd track Positions, Angles
  • For the 3rd track Positions, Angles
  • Table of number of events passing the difference cut for each track number with the ratio to the total number of events in last 4 columns.

Track       cut         XP         YP          X          Y
 1   1.0000e-04       79726       79736       79722       79718 0.9998 0.9999 0.9998 0.9997
 1   1.0000e-05       79704       79719       22415       21652 0.9995 0.9997 0.2811 0.2715
 1   1.0000e-06       71066       71935        2293        2312 0.8912 0.9021 0.0288 0.0290
 1   1.0000e-07        9519        9917         240         210 0.1194 0.1244 0.0030 0.0026
 2   1.0000e-04        7653        7655        7652        7651 0.9997 1.0000 0.9996 0.9995
 2   1.0000e-05        7650        7651        2168        1967 0.9993 0.9995 0.2832 0.2570
 2   1.0000e-06        6896        6818         254         192 0.9008 0.8907 0.0332 0.0251
 2   1.0000e-07         898         895          29          20 0.1173 0.1169 0.0038 0.0026
 3   1.0000e-04         328         328         328         328 1.0000 1.0000 1.0000 1.0000
 3   1.0000e-05         328         328          95          87 1.0000 1.0000 0.2896 0.2652
 3   1.0000e-06         296         303          12           9 0.9024 0.9238 0.0366 0.0274
 3   1.0000e-07          30          44           2           1 0.0915 0.1341 0.0061 0.0030

Compare HCANA to ENGINE for Golden track

Replay the data

• Replay the data with HCANA
  • Go to your hcana_replay directory and check that run 52949 is on the cache disk.
  • If you have compile the hcana code then you should have the environment variable HCANALYZER define, then make symbolic link : " ln -sf $HCANALYZER/hcana hcana"
  • Type "hcana" and get the prompt:"analyzer [0]"
  • type " .x replay_both.C(52949,1,100000)" to replay up to CODA event ID number =100,000
• Replay the data with ENGINE:
  • cd to the replay subdirectory in your "analysis" directory
  • Type " replay_inter grun=52949 gstop=100000" to replay up to CODA event ID number =100,000
  • Should create ntuple file paw/hms_dc_52949.rzdat . If not then check that h_dc_ntuple variable is set in the PARM_FILES/REPLAY_FARM_COIN.PARM
  • To make comparison to HCANA replay, convert the hbook file to root file using h2root. Type "h2root ./paw/hms_dc_52949.rzdat ./paw/hms_dc_52949.root"

Macro to compare

• Compare HCANA and ENGINE:
  • Go to your hcana_replay directory. Assume that you have a softlink to the paw directory to have " ln -sf $SOME_DIR/analysis/replay/paw paw"
  • Type "root"
    • .L Macros/analyze_hcana_tree.C
    • .L Macros/analyze_engine_tree.C
    • .X Macros/comp_goldtrack_engine_hcana.C("./Rootfiles/replay_both_52949.root","../replay/paw/hms_dc_52949.root")
• You will get the comparison plots shown below.

Comparison plots

• Golden track parameters parameters
• Golden track number and chi2/ndf Delta, momentum, index and chi2/ndf
• Differences fo Golden track parameters Delta, momentum, index and chi2/ndf

Compare HCANA to ENGINE for SOS drift chambers

Replay the data

• Replay the data with HCANA
  • Go to your hcana_replay directory and check that run 52949 is on the cache disk.
  • If you have compile the hcana code then you should have the environment variable HCANALYZER define, then make symbolic link : " ln -sf $HCANALYZER/hcana hcana"
  • Type "hcana" and get the prompt:"analyzer [0]"
  • type " .x replay_both.C(52949,1,100000)" to replay up to CODA event ID number =100,000
• Replay the data with ENGINE:
  • cd to the replay subdirectory in your "analysis" directory
  • Type " replay_inter grun=52949 gstop=100000" to replay up to CODA event ID number =100,000
  • Should create ntuple file paw/hms_dc_52949.rzdat . If not then check that h_dc_ntuple variable is set in the PARM_FILES/REPLAY_FARM_COIN.PARM
  • To make comparison to HCANA replay, convert the hbook file to root file using h2root.

Macro to compare

• Compare HCANA and ENGINE:
  • Go to your hcana_replay directory. Assume that you have a softlink to the paw directory to have " ln -sf $SOME_DIR/analysis/replay/paw paw"
  • Type "root"
    • .L Macros/analyze_hcana_tree.C
    • .L Macros/analyze_engine_tree.C
    • .x Macros/comp_sos_engine_hcana.C("Rootfiles/replay_both_52949.root","paw/hms_dc_52949.root")
    • You will get the comparison plots shown below.

Comparison plots

• Focal plane position and angles for events with number of tracks greater than or equal to one. The plots are 6 histograms: the top plots are X and Y ( or Xp amd Yp (mr)) with results from HCANA (black) plotted are top of ENGINE results (red), the middle plots are the difference between the top histograms, and the bottom plots are for each event the difference between the HCANA and the ENGINE with the scale of +\- 0.0001 . For the bottom plots of the angles the difference is in radians not mr.
  • For the 1st track Positions, Angles
  • For the 2nd track Positions, Angles
  • For the 3rd track Positions, Angles
  • Table of number of events passing the difference cut for each track number with the ratio to the total number of events in last 4 columns.

Track   cut             XP         YP          X          Y
 1   1.0000e-04        9732        9734        9733        9732 0.9998 1.0000 0.9999 0.9998
 1   1.0000e-05        9731        9731        9723        9720 0.9997 0.9997 0.9989 0.9986
 1   1.0000e-06        9731        9731        4253        7348 0.9997 0.9997 0.4369 0.7549
 1   1.0000e-07        9544        9613         362         988 0.9805 0.9876 0.0372 0.1015
 2   1.0000e-04         254         255         255         254 0.9961 1.0000 1.0000 0.9961
 2   1.0000e-05         254         254         254         254 0.9961 0.9961 0.9961 0.9961
 2   1.0000e-06         254         254         149         172 0.9961 0.9961 0.5843 0.6745
 2   1.0000e-07         250         249          16          24 0.9804 0.9765 0.0627 0.0941
 3   1.0000e-04          57          57          57          57 1.0000 1.0000 1.0000 1.0000
 3   1.0000e-05          57          57          57          57 1.0000 1.0000 1.0000 1.0000
 3   1.0000e-06          57          57          26          43 1.0000 1.0000 0.4561 0.7544
 3   1.0000e-07          57          55           2           5 1.0000 0.9649 0.0351 0.0877

Compare HCANA to ENGINE for raster

Before you begin, make sure the interested run is in the cache directory. e.g. run 52947.

Replay the data

• Replay using ENGINE
  • Go to the "replay" sub-directory in your "analysis" directory.
  • Type " replay_inter grun=52947 gstop=100000" to replay up to CODA event ID number =100,000
  • Should create ntuple file paw/hms_dc_52947.rzdat . If not then check that h_dc_ntuple variable is set in the PARM_FILES/REPLAY_FARM_COIN.PARM
  • To make comparison to HCANA replay, convert the hbook file to root file using "h2root paw/hms_dc_52947.rzdat paw/hms_dc_52947.root".

• Replay using HCANA
  • Go to your "hcana_replay" directory.
  • If you have compiled the hcana code then you should have the environment variable HCANALYZER define, then make symbolic link : " ln -sf $HCANALYZER/hcana hcana"
  • Type "./hcana" and get the prompt:"analyzer [0]"
  • Type " .x replay_both.C(52947,1,100000)" to replay up to CODA event ID number =100,000

Macro to compare

• To compare ENGINE and HCANA raster signals,
  • Go to your "hcana_replay" directory. Assume that you have a softlink to the paw directory to have " ln -sf $SOME_DIR/analysis/replay/paw paw"
  • Type "root"
    • .L Macros/analyze_hcana_tree.C
    • .L Macros/analyze_engine_tree.C
    • .x Macros/comp_beam_engine_hcana.C("Rootfiles/replay_both_52947.root","paw/hms_dc_52947.root")
    • You will get the comparison plots shown below.
      • raster x signals [[1]]
      • raster y signals [[2]]

Hall A Online GUI software

• The code and documentation are available at this public directory.