
	ADD Review II: Thursday 27th April 2000
	
Participating:
	Allan Hornstrup
	Peter Kretschmar
	Stefan Larsson
	Niels Lund
	Sami Maisala
	Carol Anne Oxborrow
	Niels Joergen Westergaard
	
Agenda: 

1) Rejected RIDs: RID 34 - multiple material attenuation extensions
		RID 45 - Use of ISDCLVL designations throughout document
2) Open RIDs: RID24 - GUIs ROOT and interactive executables
3) Open RIDs: RID27 RID28 - Accessing -MOD extensions
4) Open RIDs: RID32 - obsolete data structures in j-performance and 
		j-known-src-calib
5) Open RIDs: RID40 RID41 - Feasibility of j-known-src-calib
6) Open RIDs: RID47 RID48 - moving/discarding j-calib-adc-cor
7) Open RIDs: RID43 - keep j-cor-evt-evaluate as separate executable?
8) RIDs with no answers: RID6 and RID25 need an answer from Sami Maisala.
9) General issues: using OGIP standard keywords in data structure templates
10) General issues: using -INT data extensions
11) General issues: should j-cor-gain-temporal and j-cor-gain-spatial be kept
	separate?
12) Accepted RIDs: are there any accepted RIDs that should not be accepted?
13) AOB

See also: JEM-X ISSW Architectural Design Document. Review II. Draft: 26/4/00.
	The decisions and results of this meeting can be found in version 1.0
	of the Review II document which will be available within a week.

**** Logging Started : Thu Apr 27 09:02:44 MET DST 2000
<larsson> Hi CA and good morning. END
<peter> More or less, my brain is still gearing up :-) END
<peter> log #jemxadr JEM-X_ADR_2
> I thinkyou need a slash before that peter, otherwise it just gets sent as
<allan> Don't worry, we'll log the ADR. END
> a message END
> Let's wait another 5 minutes for any non-SDASTers to turn up END
> Does everyone agree with the agenda: I've tried to break the RIDs up by
*** njw (~njw@heao1.dsri.dk) has joined channel #jemxadr
> subject. Some issues are big enough to warrant further discussions in other
> meetings, and maybe protracted discussions with ISDC ;-) END
*** smaisala (~smaisala@mizar.astro.helsinki.fi) has joined channel #jemxadr
<allan> Could we set off with a logistic problem: 
> Hi Sami! good to have you with us, I got your reply to Reiner's last RID END
<allan> I suggest to end today at lunch time, i.e. 12.30.
<smaisala> Hi all! END
<allan> The reason is that I have another meeting this afternoon,
<peter> I _hope_ strongly that we can finish by lunchtime END
<allan> where my precense is required, and I also think that more than 3-4 hours
<allan> on IRC is a bit on the long side. Thus, if we dont finish before 12.30,
<larsson> I will have to leave at 10.45. END
<allan> we may continue tomorrow.  OK?
<allan> END
> Me too! That means lots of AI's! And possibly further chat meetings about individual topics raise by the RIDS END
*** niels (~allan@apollo.dsri.dk) has joined channel #jemxadr
> Hi Nils! 
<niels> Hi
> I forsee points 3,5,6,7 needing further discussion END
> Okay, it's 9.15 - lets get a start on point one: rejected RIDs, first RID
> 34: I for one support NJW's position here and agree with his rejection. Any
> oppositng views? END
<allan> NO END
<peter> No objection END
<smaisala> No END
<njw> good END
<larsson> Were do I find NJs replay?
> In the RID document I sent to you yesterday afternoon, I'll send it to you
> now, if you don't have it. END
<larsson> I found nothing there (maybe I printed wrong version?) END
> RIDdoc.ps just sent to you - you'll need it for the rest of the meeting. END
> fOkay RID45: I accept my rejection, what about the rest of you, esp. Peter? END
<allan> If you don't get it in the mail, we can post it on the web END
<peter> Here I (naturally) disagree:
<peter> The science analysis levels are not volatile, they will be an important feature
<peter> of ISDC's design for Scientific Analysis.
<peter> This is to say that we might change a name or two, but the principle of named levels will be kept.
<peter> IMHO it is important for the integration of our ISSW code into the ISDC framework that we know at
<peter> which "level" each part of the ISSW is placed. END
> You know that from The Figure. END
<peter> Well, I prefer redundant information over a possible lack of the same. END
> But what we've provided is all that we know. Also what about the questions
> about SWG1 and SWG2 - surely they we an attempt to sort data structures into
> processing levels. What happens to them now? END
<allan> Peter, I think we need one more argument to have redundant information END
<peter> The enumerated SWG levels were the first idea. 
<peter> Works well for the ScW pipeline (up to OSM) but not really beyond.
<peter> They are not completely eliminated because quite a bit of existing V1 software 
<peter> works on the assumption that they exist. 
<peter> But I can live with being outvoted on this END
<allan> OK END
> Can I put an AI on Peter and ISDC to provide a document (even a quite short
> one) that explains: a) what ISDC levels are b) what they will be used for
> c) how they will co-exist with SWG1 and SWG2 d) how data structures will be
<peter> Hey, I tried to do that at our last meeting. 
> divided between levels (esp. those that get used many places) e)everything
<peter> And they will be definitely mentioned in the reworked Anlaysis ADDs (OSA, QLA, SA). 
> else that's been decided thus far about ISDC levels END
> Yes, if the information, approved by ISDC, is forthcoming, then I'll
> incorportate it into the ADD . END
<allan> Seems that we are approaching an agreement
<allan> end
> If Peter will provide the information in concrete terms: perhaps including a
> couple of examples of the placement of data structures into the level
> system, then I'
> ll implement it as best I can: this may lead to updates of EVERY COMPONENT:
> so be warned. END
> Due date on that AI Peter? END
<peter> Publication of (updated) analysis ADDs - say end of May. END
> So I can't start updating our ADD till end of May? END
> I need your ISDCLEVL description first END
<peter> Sorry, I misunderstood your intention.
<peter> But it doesn't make too much sense if you try to change this.
> Send me an ISDC level description document and I'll even put a short bit
<peter> It would have to happen at the level of major components, like j_data_binning.
> about it in the introductory chapter. END
<peter> OK, let's do it this way:
<peter> I send you a description of what is meant by these levels for the introductory chapter
<peter> But I do not insist on the mentioning of levels in the drawings and descriptions of
> Covering all the points litsted above, I hope END
<peter> individual components. Though it would be nice. END
<peter> Well, I hope it will do. END
> Okay, Review Decision will read: `Peter Kretschmar will provide an
> introductory description of ISDC processing levels, from which CAO will
> update the ISSW overview and data dictionary.' Is this satisfactory? END
<peter> Yes. END
> Due date for your piece Peter? END
*** Signoff: allan (Ping timeout)
<peter> May 5. END
<niels> (Allan: I am still listening from here, my other computer died :-( )
> Okay, Open RIDs: RID 24 on Helsinki 
> My feeling is that for all the interactive components it isn't quite clear
> where the interface with ROOT and OSMS lies, or to what extent GUIs will be
> written/developed by us. Could we perhaps do with a couple of chats, or
> even in-person meetings to clear up some of these issues?
> Also, do we all need an introduction to 
> ROOT and OSM? END
*** allan (allan@ah-notebook.dsri.dk) has joined channel #jemxadr
<peter> The problem is, that is it exactly this border that should be defined by now.
<peter> We have been discussing this more than once and Sami has been at ISDC inbetween.
<peter> There should be a real effort to understand the possibilities of the available tools
> Agreed END
<peter> and then a clear decision if - with current knowledge - we can accept working with them or not. END
<smaisala> The main problem is lack of documentation in OSM and ROOT. It is impossible to make any   
<smaisala> try to use all capbilities what ROOt and OSM can offer when there is no documentation. END 
<peter> I agree that documentatoion is somewhat short, but that's why I've requested since a long
<peter> time that this is simply discussed directly with Reiner.
<peter> It worked nicely for OMC this way. They came, had a demonstration of the capabilities
<peter> and saw what could be done and what can't.
<peter> I should qulaify my statement above - ther is documentation, it's just quite opaque. END
<peter> My problem is that the work of having these discussions wasn't done yet.
> Could we organise an OSM/ROOT meeting at ISDC? A ROOT workshop? END
<smaisala> That sounds nice. END
<peter> It could be done, but I'm not sure it is required for this decision. END
> I'd be very glad to learn all about both systems, what about you others? END
<allan> Me too, but I also agree with Peter, 
<peter> To learn _all_ about ROOT would take more time than you probably want to invest :-) 
<allan> that accepting the RID could be done END
<peter> For OSM there isn't too much to learn. END
<peter> Regarding the RID: I am gladly willing to answer questions (or forward them to the knowledgeable people)
<peter> about the inherent capabilities of the existing software. END
> Shall we accept the RID then, with Sami getting the information he needs from
> Peter to make a complete update of the description of this component? END
<smaisala> I agree. I will ask all those questions. END
<allan> y end 
<peter> OK with me. END
> Also, can I put an AI on someone me, Peter, Allan(?) to organise an OSM
> (and ROOT) workshop at ISDC? END
<allan> <let's us rather phrase it to look into the possibility and
<allan> feasibiltiy of such a meeting. END
> That's a start, at least. Peter willyou take that AI? END
<allan> Put me on that as well, we need to discuss the feasibiltiy END
<peter> Fine with me. END
> Due date? END
<allan> June 15? END
<peter> OK END
> okay END
<allan> On to 3) ? END
> Review decision: `Accepted: SM to re-write component description with help
*** eesko (~eesko@aquarius.astro.helsinki.fi) has joined channel #jemxadr
> from PK concerning information about the exact interface with ROOT/OSM' END
> Is that satisfactory? END
<allan> y end
<peter> Y End
> Onto point 3) - acessing -MOD files: this opens a whole can of worms, namely
> where will calibration and instrument model files be, how do we access them,
> and how do we ensure that we're using the most up-to-date files? END
<peter> Let's separate this into two cases:
> While the RIDs are only filed for a couple of components, this issue
> affects many components. END
<peter> (a) -MOD structures:
<peter> These won't change often and we have many of them for some components.
<peter> Followin an idea of NJW, I propose to have a single MOD-group.
<peter> IT is up to the ISDC Analysis system to ensure that a MOD-group with the correct -MOD children
<peter> exists when the analysis executable is called. 
<peter> In Offline Scientific Analayis the user can modify this group before calling the analysis scripts. 
<peter> (b) -CAL and -CFG (="configuration" = instrument information not based on cal. data)
<peter> Here ISDC plans to have large Index groups for each such data structure which administrate the various instances
<peter> possibly over the whole mission.
<peter> From the ISSW point of view, it would be usually easiest to assume that a single instance is available and that the
<peter> DOL to this instance is passed as parameter to the executable.
<peter> All this assumes that usually an executable needs at most a few of these data structures.
<peter> Selecting the 'right' instance is again a task for the Analysis System supplied by ISDC or the user in OSA. END
> I think I should perhaps write all that in the introduction to the data
> dictionary: and all people with components using -MOD -CAL and -CFG files
> should update their components accordingly (I know I have quite a lot inthis
> area that didn't attract RIDs, but perhaps should have). 
> NJW and SL, will you accept the RIDs and update your components according
> to this information, that includes diagrams? END
<larsson> OK. END
<njw> Y
<peter> Fine END
> Review decision: `Accepted: this component, and others using -MOD, -CAL,-CFG 
> data structures to be updated to indicate where these files come from.' END
> Next RID32: Sami and Juhani and Stefan, what data structures should I put
> in place of the ones that Stephane claims are obsolete? I'm stumped on this
> one END
<peter> If I may intervene:
<larsson> You may Peter...
<peter> JMXi-CYCL-CNV -> JMXi-CSSW-CNV  (this is the normal HK)
<peter> JMXi-DRV1-CNV was a placeholder for "derived HK parameters", e.g. the difference between two voltages.
<peter> Since we do not intend to have a progranm doing such calculations (at least at the moment) -> delete
<peter> JMXi-SPR1-SCP was (probably) a placeholder for so-called "science parameters", i.e. HK-like data
<peter> derived from actual science data. Currently we do not have such a thing, but we may eventually under a different name.
<peter> JMX-SRLC-SFR -> JMXi-SRCL-RES END
<allan> It seems, that the two structures we may
<allan> delete will have impact ionly on the perfrmance compoinents 
<allan> END
<smaisala> Thanks Peter. That was nice presentation. :-) So I think we can remove those two data structures.
<allan> If the "derivation of HK parameters" is needed, it should be 
<allan> done within the perf-components, OK ?
<allan> ENd
<peter> I agree. END
<smaisala> Yes. That can be done by jperformance. END
> So Sami and Stefan must update their components with new data structure
> names and deleting two data structures, and I will remove/rename the
> data structures in the data dictionary, Okay? END
<allan> y
<larsson> OK. END
<smaisala> ok END
> Review decision: Accepted, CAO ro rename/remove relevant data structures, and
> SM and SL to update the components that used the obsolete data structures.
> Okay? END
<allan> y
<larsson> OK. END
> Next, RIDs 40 and 41: feasibility of j-known-src-calib. Soeren has serious
> reservations here, and I'm quite interested to know how this will be done
> myself. Helsinki? END
<smaisala> We have discussed this earlier. The problem is simple: Is it possible to find well defined peak in the spectrum of the source?
<peter> Difficult. Maybe a single peak at the iron line for some sources. END
<allan> I agree with Sami. THis has
<peter> But this would always be on a strong continuum. END
<allan> been discussed loosely many times, and now is possibly the
<allan> time to say, that we don't think it is feasible.  The component
<allan> got in as a replacement of handling the calibration from the at that 
<allan> time on-board source in the mask.
<allan> We have kept it as a placeholder, to not rule out the
<allan> possibility to do inflight calibration like that.
<allan> Maybe we should now leave this option to "offline analysis".
<allan> END
<peter> My proposal is an action on Soren (and maybe Allan) to define:
<peter> - if something like our j_ksc... is possible
<peter> - if yes: what a sensible procedure would be. END
<allan> OK ENd
> OKAY, AI on Allan and Soren and Me to discuss this problem during one of
> our monday software meetings (I've agreed this with Soren). would that be
> all right? END
<peter> OK with me END
<smaisala> yes END
<allan> Actually, I think our Helsinki friends should provide some input
<allan> before that meeting, just to let us know how far they
<allan> have looked into possible sources etc.
<allan> But I guess we just take that as part of our action 
<allan> to get that info. END
> AI on Helsinki to send us ALL their info on this procedure first? END
<allan> No, dont make that a formal action, we just talk to them before our meeting END
> Okay. END
<smaisala> OK END 
> Next, RIDS 47 and 48: can we get away with removing j-calib-adc-cor (and by
> implication j-cor-sci-adc? I'd like to talk to NL first about this, but I
<larsson> Sorry folks but I have to leave now. I will read through the disc.
*** nl (~nl@lin1.dsri.dk) has joined channel #jemxadr
> feel confident that instead of having nominal binning files and having
<larsson> ' this afternoon. Bye. END
> the IT send corrections to produce the -CFG files, it would be easier for
*** niels has left channel #jemxadr : (niels)
*** Signoff: larsson ()
> them to provide the -CFG files directly. This would happen very rarely, if
> at all. END
<peter> In which case they might be named -MOD ... but otherwise I agree. END
<allan> I agree too.  END
> -MOD is a good name too ;-) The thing is these components have changed from
> being corrections to individual events to corrections to the binning limits
> tables, so no SWG processing will be done. END
<peter> OK, let's say if NL confirms that updates will be rare, we drop j-cor-**-adc.
<allan> y
<peter> and I will check with Stphane if the data structure should be called -MOD or -CFG. END
> Review Decision for RIDs 47 and 48:' Components j-calib-cor-adc and j-cor-sci-adc
> will both be omitted in favour of the IT providing directly the effective
> energy binning tables ****-MOD for each telemetry binning configuration',
> pending NL approval. END
> Sorry to back up here, but RID40 was not about gain calibration feasibility,
> but whether corrections to pre-flight and j-auto-calib calibrations should
> be done on raw data, and so be a complete, free-standing correction, in it's
> own right. Or should corrected data be used so that the corrections found
> by j-ksc are finetuning corrections that are applied after j-correction
> corrections? END
<allan> Soerens comment is correct, the component should 
<allan> include the raw data - at least for comparison. END
<allan> (i.e. irrespective of the final output is corrections or 
<allan> finetuning corrections). END
<nl> I can confirm that updates to the binning tables will be rare. END
<peter> Thanks for the input Nils! END
<nl> Unfortunately I will have to leave now for another JEM-X meeting (on the User Manual) END
<peter> Regarding RID 40: I fear the answer to that one ties in with the AI to define the whole scheme.
<nl> quit
<allan> I agree with Peter END
*** Signoff: nl ([=V97=] Leaving)
<peter> But probably it is a good idea to start on raw data. END
<allan> y
> So in j-auto-calib, people should choose between using pre-flight and
> Sorry, I mean in j-correction, people must choose between using pre-flight
> and j-auto-calib corrections, or in-flight KSC corrections. If KSC spatial
> gain is not feasible, then j-auto-calib corretions will be the only gain
> corretions available, and only position corretions will have a choice of
> pre-flight or in-flight corrections. correct? END
<peter> Not necessarily. We should have spatial gain variation data from the ground calibrations.
<peter> You just would not have in-flight  data. END
<allan> Well, as I said, we may have some off-line analysis results, which may be 
> Yes, but no need for folk to choose between pre-flight and in-flight
> corrections. END
<allan> written back in as in-flight calibration END
> So I should keep all correction options open in j-correction? END
<allan> currently, I think so. END
> Okay, Sami, do you accept RID40? Will you start from raw data instead? END
<smaisala> Yes I can start from raw data. END
> Next, RID 43: should event evaluation be a separate executable, or should
> this be done within each executable of j-corretion? END
<allan> I support the latter END
> Note: the `accepted' here only applies the change of the keyword name END
<peter> I'm also in favor of the latter.
> I'll look into merging event evaluation into individual corrections. So 
> that `accepted' becomes a global `accepted'. END
<peter> If we define EVT_EVAL or STATUS as essentially a bit-flag rececptable, then each correction executable
<peter> could add specific numbers (1,2,4,...) to this column.
<peter> In automatic analysis we might exclude everything where this is not zero.
> Now, RIDS with no answers: RID 25 is very much like 24 and probably should
<peter> In interactive work a user might select specificly. END
> get the same review decision. Sami has sent an email answer, so I'll have
> a look at that now. END
> I agree, Peter. That's just how I envision things too. END
> Okay, Sami accepts RID25. What about RID6 Sami? END
<smaisala> I was weird question to mee because I haven't draw drawn that picture. But if I understood that correctly
> Sami, will you update j-performance and j-k-s-c to indicate which tasks are
> performed manually? END
<smaisala> I have to specify what parts of applications are used manually and what are automatically.
> Okay, so RID6 is accepted. Good. Now onto the the question of OGIP standards.
> SB has vented some spleen here, concerning naming of columns in FITs
> extensions and nameing of keywords. Basically, I suggest that we ALL have
> a look through the OGIP standards (AI on CAO to send URL to all SDAST for
> OGIP standards) and ensure that our columns and keywords conform to these
> standards? Any revolutionaries out there want to propose something 
> different? END
<peter> Soren is right. We probably have overlooked some places where OGIP standards, could/should be applied.
<peter> But I warn that in some cases one can discuss if an OGIP standard actually applies.
<peter> Nevertheless, the exercise is probably very useful. END
> Any more comments on OGIP? END
> Next, general issue:  -INT extensions. It seems to be generatlly agreed that
> these should be under configuration control, complete with approved template
> files and the whole shebang. The other question is whether they should be
> used at all. Comments? END
<peter> In some places, e.g. the intermediate stage of j_correction, they could be quite useful.
<peter> In other places, they seem superfluous.
<peter> I agree with Stphane's basic comment: if we do not always delete them, they should not be called -INT.
> The next point concerns j-correction specifically and I intend to remove that
> INT extension. END
<peter> So instead of JMXi-SHAD-INT we should probably have something like JMXi-FULL-SHD and -REST-SHD.
<peter> (Note that ISDC uses -SHD to denote shadowgrams at least in one other case).
<peter> Were we keep -INT structures it would be best to still have templates and all for them. END
<smaisala> Yes I agree. I can remove all -INT structures END
> So we're agreed? INT structures are out. END
<peter> Anybody else with an opinion? END
> Okay, AI on all to remove/rename/redesign -INT files. END
<smaisala> OK END
> Next, although this has not become an official RID, it has been suggested to
> merge j-cor-gain-temporal and j-cor-gain-spatial, which would amongst other
> things, allow me to get rid of one INT  extension. Also, SB has pointed out
> that,  j-cor-gain-spatial many not act merely on imaging events but also
> on none-imaging events to make them compatible with imaging events that have
> been corrected for spatial gain variations. Spatial gain corrections wil
> also use on-ground correction tables, and so will run even when there are
> not KSC corrections available, which increasingly seems to be the case. 
> Any objections to merging the gain correction executables? END
<allan> I have no objections to a merge. END
<peter> No objections
<peter> END
> Next question, should j-cor-gain-spectral also be merged with these two,
> since it requires reading in exactly the same correction tables as for
> corrections on other types of data? These means we would be left with only
> two correction executables: one for correcting positions, and one for 
> correcting energies. A neat division of labour or gnat-brained stupidity? END
<peter> You could, but in other places we have different executables for the spectrum mode and the events. END
<peter> While youu read in the same correction tables you work on different data. 
<peter> But I have no strong feelings either way. ENND
> The thing is, spectrum mode is not spectra, it's events that only have
<allan> Me neither, although we have tried to keep an as modular development as 
<allan> possible. END
<allan> i.e:  You can merge them, if that's most feasible. END
> energy values.
> So I'm not rebinning spectra, which is different from correcting events.
> I'm correcting events for which we have not position data END
<peter> Wait: Spectrum mode _is_ spectra! Timing mode is just time stamps of events! END
> Sorry, you're right, I'm wrong. These ARE spectra: I'll keep the executable
> separate. Sorry!
> That's nice because, I've already written a spectrum re-bininng function
> fro j-calib-gain-fitting. So this one's already done. False Alarm. We now
> have 3 correction executables. END
> Finally, I've taken the liberty of writing review decisions of a positive
> nature for every RID where the author reply seemed to indicate acceptance of
> the suggested solution. Are there any case we're I've overstepped my
> authority and written that people will perform changes and corrections
> they have no intention of doing? END
<allan> (i.e. Peter, do you accept your own RIDs?) END
<peter> Yes, I do accept my own RIDs - grudgingly ;-) END
> That's understandable, I hear that Peter Kretschmar is a really tought
> critic ;-) END
> Any one else with problems over my one-woman judge-jury-and-executioner
> procedure? END
> Okay: now for the bad news. Every single RID (except for the two rejected
> RIDs) is an implied Action Item. I will not add each of them to the AI list,
> but you are just as responsible for performing the actions agreed to in the
> RID decision. When is an appropriate deadline for the completion of RIDs?
> 15th June 2000? END
<smaisala> I agree END
<allan> Seems OK, just in time to have the updated ADD ready to the SDAST meeting in Stockholm ? END
<peter> This would make it after the ISSW delivery. But OK. END
<peter> ISSW V2 delivery, to be specific. END
<allan> Fine, then CA can concentrate on that and not on ADD END
> No, I'm on holiday June 7-21st and I'll have to wring all the updates out
> of you. Let's make the deadline for RIDs June 1st instead if we want the
> updated version ready for the meeting. END
> I 0.5 to 1.0 weeks to get input from the rest of SDAST and at least 1.0 
> weeks to integrate it. END
<allan> We need an answer - is June 1st OK? END
<peter> Fine with me. END
<smaisala> Ok END
> That's an AI on everyone to have their RID actions finished by June 1st.
> I would like to have input by then, instead of everyone waiting till after
> the deadline to even start editing. I s this possible? END
> So  AOB? END
<peter> Not from my side. END
<allan> n
<smaisala> n
> One last note: can we in future assume that if I don't send an agenda for
> a Tuesday morning chat, then there's no chat. If I do send an agenda on
> Monday, I'll try to do it before lunch, and the chat will be at 9.30 Danish
> time the next day. Is this okay with you all? END
<allan> good idea ! END
<peter> I agree! END
> If there are no more comments: many thanks for a productive, and quite
> speedy review (we got through 13 points in 3 hours!), and  may you all
> have a sunny and relaxing weekend. END
<peter> Same to you. But I'll have to work this afternoon and tomorrow first ... don't know how that is in Denmark ;-) END
> We've got sun - and I intend to enjoy it. Bye all! END
