
	Weekly chat meeting #34: Tuesday 15th August 2000
	
Participants: Peter Kretschmar, Allan Hornstrup, Sami Maisala, Carol Anne
	Oxborrow, Niels Joergen Westergaard
	
Subjects discussed: 	Status of V2 deliveries
			Summary of V2 integration
			Bugs in DAL3JEMX
			Sharing Test data
			Sharing delivered source code
			Problems with coding/contact in Stockholm

Action Items: None


**** Logging Started : Tue Aug 15 09:28:12 MET DST 2000
*** peter (~pkretsch@isdcul10.unige.ch) has joined channel #jemxadr
<peter> Good mornin Carol Anne!
*** smaisala (^smaisala@mizar.astro.helsinki.fi) has joined channel #jemxadr
*** allan (~allan@apollo.dsri.dk) has joined channel #jemxadr
> Morning guys - please forgive me being late - I'm having an interesting
> exchange of emails with a guy at ESTEC who thinks I'm complaining about
> something, when I'm actually trying to save him a little work! END
<peter> Planck or JEM-X related? END
> Planck END
> I'll try and rustle up NJW END
*** njw (~njw@heao1.dsri.dk) has joined channel #jemxadr
> He's coming now. What about the Swedes? I can't get hold of them by email
<njw> Hi, sorry I'm late END
> or by telephone the last couple of days. END
> I suggest we start without the swedes: I've tried stefan's number again
> and there's no answer. END
<peter> I think, he's still on holiday and I'm not sure Sara has been introduced to this kind of chat. END
> That would explain alot. Okay let's here where we've each got to.
> You first Peter. END
<peter> OK:
<peter> As you know, j_bin_evts_spectra has been delivered long ago.
<peter> j_bin_spec_spectra is finished, waiting for templates to become official.
<peter> j_bin_bkg_spectra was 'finished' yesterday, i.e. it builds but hasn't been tested at all yet, so it will probably need another full day of work and might  be delivered by the end of the week. 
<peter> Jan Soldan has finally moved here, the administrativbe  troubles aren't all resolved (they haven't seen any of the promised money yet) and thus no real progress has been made on the j_bin_***_lc executables.
<peter> END
> Thanks Peter. What about you NJW? END
> I'll just go rustle up NJW again ENd
<njw> I've delivered the  shadowgram and basic_recon executables and am
<njw> starting on src_find (I have an IDL program running so it should not be too hard)
<njw> I have also given test data to Stefan for the source parameter extraction
<njw>  - hopefully we get something before the Alicante meeting. END
> When do you think src_find might be deliverable? END
<njw> In two weeks or so END
> Okay, Sami, you're turn to tell all END
<smaisala> Ok. I have delivered j_bkg_spat_sel and j_bkg_spec_sel. j_bkg_scal_calc is under work.
<smaisala> I yhink it is going to take about a week to deliver this component. I have had some problems
<smaisala> with DAL3AUX functions. END
> So basically you'll be finished by the end of August? That's good. eND
<smaisala> Yes. END
<smaisala> If you need to use DAL3AUX functions you MUST check that you had all /isdc/scripts/login file 
> Okay, so my turn: I've delivered j_cor_gain_events and j_cor_position, and
> j_cor_gain_spec will be delivered this week. After this I'll be working on
> setting up a CVS-type system for the ADD. END
> Thanks for the tip, Sami. END
<smaisala> The important line is setenv AUXL_REF_DIR          "/home/astrox/paltani/ISDC_REF"
> Unfortunately it's anyone's guess how stefan's team is doing - they have
> so little experience with DAL that is'
> it's no wonder they threw in the towel. I've offered several times to go
<njw> I know of one problem that Sara had (and I have the same problem)
> up there next week and help them out, but I've had no reply whatsoever END
<njw> There are difficulties with the DAL3JEMX functions that supposedly should
<njw> be an easy way to get hold of events and do some clever selection. But it
<njw> does not work. Has anyone worked with DAL3JEMX ? END
<peter> Yes, I think succesfully too :-) END
> No I've avoided all that fancy stuff - the more layers of other people's
<peter> It would probably help if such questions were circulated among SDAST, especially with me.
> functions there are the more difficult it is to see what's going on.. END
<njw> OK, Peter, will you please take a look at j_ima_shadowgram_work.c and
<njw> tell me what I'm doing wrong ? END
<peter> Any hint of what is going wrong? END
<peter> I agrre, btw, that the DAL3JEMX interface to events is somewhat quirky. But the powerful machine behind it saves you from a lot of trouble once your data is less then perfect. END
<njw> Have a look at the RILmessage.log from the test runs. It should be in there.
<njw> But I've neglected a malfunction (if wrong, continue anyway) END
> Could I ask everyone to send me a tarred, gzipped file containing all 
> their source code  for delivered modules so that I can put it on the
> developper's forum. That way we can take a look at each other's code
<njw> Didn't Peter send you j_ima_shadowgram and j_ima_basic_recon ? END
> and see how other people tackled various problems: I hope this will be
> useful.
> END
> No, I expect the individual developpers to do it for each delivery END
> Also if you could send me test data files you've created for the various
<peter> To NJW: no I did not. And funnily enough, I do not have trivial access to delivered source code. END
> exxecutables, that could be useful: maybe save other people time by having
> them re-use other people's test data. My own experiece is that making
> test data takes AGES END
<njw> To Peter: should I send the code to you as well ? END
<peter> Yes, that would be faster. END
> Also, I'd be very grateful if you folks would send me an email as soon as
> you've delivered something, saying when, what and version number so that
> this information can go straight into the monthly report. END
> Could you do this Sami - we have no mention of your deliveries yet. END
<smaisala> Of course. Actually I have delivered those components about a month a go. END
> So why wasn't it in July's report??? END
<smaisala> Sorry, It must be my mistake :( 
> Okay, send that email, and we'll put it in this month's report. END
> Now I assume we'll all spend the rest of this month completing our deliveries.
<smaisala> I was thinking that when I deliver something to ISDC it is automatically in monthly report END
> Next month, September, we'll start on regular chat meetings gain. Is that
> a good idea? END
> Thanks for the code files NJW, I'll put them on the forum right after this
> chat END
> Okay, any other business? END
<peter> I could say some words about inetgration, if youa re interested. END
> Yes, if people are still here.....I'd like to hear about integration END
<njw> Y
<smaisala> Y
<peter> OK. I'll try to summarize.
<peter> To integrate the JEM-X ISSW, ISDC will set-up a special user account under which another copy of the ISDC Environment will be installed.
<peter> The reason for this is that /isdc/dev/ is too volatile, but if we restrict ourselves to the formal Reference Platform we could not quickly have updates of, e.g., DAL3JEMX if needed.
<peter> The idea is that I manage a list of support-sw and ISSW with version numbers and anytime something changes, ask the Configuration Manager to rebuild the whole setup as on my list.
<peter> Naturally this only works on delivered items :-).
<peter> Within this environment, I will then write ROOT scripts, using ISDCTask to call executables.
<peter> These scripts will have their own parameter files and we should probably begin to discuss these parameters.
<peter> Examples of such scripts would be j_correction, j_data_binning, ...
> You mean there will be other parameters than the ones we already supply
> in our ISSW? END
<peter> The idea is rather to have a _subset_ of those parameters. One can often assume that for normal processing a user
<peter> will _not_ change a lot of parameters from their default settings. 
<peter> In addition there will probably be parameters that drive the behaviour of the script and have no direct counterparts in the executables.
<peter> A good example is that the scripts will work on Observation Groups (looping over the individual Science Windows) and thus have an OG paramter. END
> So there aren't any mandatory parameters that have to appear now in our
> *.par files to get the script to work, and haven't found their way into
> CTS yet? END
<peter> No I do not think so. While it is nice and good practice that we have common parameter names within the JEM-X ISSW, the scripts could in principle handle all sorts of choices. END
> That's good to hear END
<peter> Around the individual scripts for analysis steps, I will then also generate (a) master script(s), that will run the whole analysis for SA or QLA. 
<peter> These master scripts will then be called by the pipelines. The latter 'only' care about resource allotment (diskspace, CPU power).
<peter> It is hoped, that the master scripts will be flexible enough to be also used interactively. END
<peter> According to the current ISDC schedule, I should have the first versions of all these scripts by early October.
<peter> Please note that this means I must have seen all executables long before that since I am sure (from experience) that
> That sounds pretty ambitious peter END
<peter> every single one of them will have to be modified at least once to work together in the overall framework. END
> We're trying our best here - certainly it sounds like all the chat
> participants here today will be finished quite soon. The real question
> is when the Swedes will be finished. I'll try getting hold of Sara and
> Luis again, and repeat my offer - but perhaps they're on holiday too. END
<peter> Btw, I then have up to March 2001 to consolidate the scripts. END
<peter> Personally I do want to be able to run a JEM-X analysis Demo (with OSim data as input) at the Consortium Meeting in December.
> That's probably just as well! Do we have upto March 2001 to consolidate
> our V2 executables? END
<peter> But this demo does not have to include all branches. END
<peter> In my opinion the executables will continue to evolve even past this date. 
<peter> But naturally, the two consolidation efforts go hand in hand. END
> Good - I'm glad you
> re prepared, psychologically and otherwise for that END
> Is there anything other than making deliveries that we can do to ease the
> integration process? END
<peter> My main problem (as always) is that I feel I could use more time on this in order to do a better job. END
<peter> Easing the integration process:
<peter> - Make full use of jemx.h and the list of common parameter names
> I think we all have that feeling.....it's part of being alive as far as
> I can tell! END
<peter> - Increase exchanges of test data, where applicable, so more problems are caught even outside of what I am doing.
> THat's a good suggestion Peter  END
<peter> I should say that this "integration account" would also be a sort of repository for both 'correct' versions of the delivered ISSW and test data. 
<peter> But I still have to figure out how it all will be set up and how you best access what is there. END
<peter> Sorry - complete backtrack to something at the beginning:
<peter> Niels Joergen, is it possible that you had a problem with DAL3JEMX for JEMX-2?
<peter> Based on Sara's problems I did find a typo in DAL3JEMX which led to no JEM-X 2 events being found. END
> Are you in contact with Sara? Is she around at the moment? I'd like to get
> in contact with anyone in Stockholm (well, almost anyone) END
<njw> 1) I'm not sure whether it was only for JEMX2 - I'll check
<peter> The last email I got from her was August 9 ... :-( END
<njw> 2) I was in cantact with Sara last week
<njw> END
> With reference to Peter's suggestion that we share test data so that each
> executable gets a fuller workout before delivery, I'd like to remind you
> all that the developper's forum page has lots of fake testdata we each can
> use to flex our executables on. END
> Also, you should send me tarred, gzipped files of your test data that was
> delivered with any executables. END
> Any other suggestions Peter? END
<njw> My experience with 'old' test data is that some column names or keywords
<njw> change so that you can only use the mose recent test data END
> Test data from the current round of deliveries should work okay. Steer clear
> of everything created with Reference Platform 1.2 unless you enjoy 
> core dumps. END
> Is that the end of your integration synopsis Peter? END
<peter> I think so. Do you have more questions? END
> I can't think of anything at the moment - but something's sure to occur
> to me before October. END
<peter> In general, I feel we should intensify contacts again, but that will probably happen automatically as I try to piece together our codes. END
> As I said, we'll start regular chats again in September, when everyone's
> finished with holidays - would people prefer the next chat to be next
> week? END
> Okay, Any other Business? END
<njw> N
<peter> Not from me.
> Nor me.
> END
*** njw has left channel #jemxadr : (Bye)
*** Signoff: allan ()
<peter> Good bye! END
*** Signoff: peter (ircII/tkirc)
> Chat with you all again in September END
*** smaisala has left channel #jemxadr : (bye)
