
	Background Problem Chat#2: Friday 29th January 1999

Participating: Peter Kretschmar, Carol Anne Oxborrow, Niel Joergen Westergaard,
	Allan Hornstrup, Stefan Larsson, Niels Lund, Gerry Skinner(part-time)
	and Juhani Huovelin
	
Subjects discussed: Agenda for SDAST meeting Febuary 1--2 1999, the principal
	subject of the meeting to be background handling problem.
	Countrate data
	Background handling discussion document


Action Items: None.

*** IRC session log started at Fri Jan 29 09:27:01 1999.
#jemxadr> Hi, Peter! It's REALLY cold here! END
*** njw <~njw@heao1.dsri.dk> joined channel #jemxadr
#jemxadr> Hi NJW - good job on the background problem: even I can almost understand what's going on END
<Peter:#jemxadr> Hi Carol Anne, around here we had quite a bit of snow even down to the lake.
<Peter:#jemxadr> REALLy cold means how much, btw? END
<njw:#jemxadr> thanks Carol Anne - actually most of the discussion comes from Helsinki END
#jemxadr> Enough to make your bottom cold for a couple of hours after cycling to work! END
#jemxadr> No snow yet END
H njw          #jemxadr 0 Niels Joergen Westergaard         ~njw@heao1.dsri.dk
H oxborrow     #jemxadr 0 Carol Anne Oxborrow        ~oxborrow@130.226.216.181
H Peter@       #jemxadr 0 KRETSCHMAR Peter         ~pkretsch@isdcul10.unige.ch
315 apollo.dsri.dk * End of /WHO list.      
*** huovelin <~huovelin@taurus.astro.helsinki.fi> joined channel #jemxadr
#jemxadr> Hi Juhani - good job on the gackground discussion document. Also, thanks for the list of refe
+rences, I got most of them. END
*** allan <~allan@apollo.dsri.dk> joined channel #jemxadr
#jemxadr> Hi Allan! END
<allan:#jemxadr> Hi
#jemxadr> I hope Stefan will be with us soon. He just sent me an email to say he has an allergy to pres
+ervatives, so we'll all be eating very healthy, home-cooked chemical-free food on monday evening. Be w
+arned! If any of you have a miniumu daily req
#jemxadr> uirement for additives in your food, I can always throw in a few bowls of potatoe chips! END
#jemxadr> We'll wait one more minute and then get on with discussion of the agenda for Monday's meeting
+ END
H allan        #jemxadr 0 Allan Hornstrup                ~allan@apollo.dsri.dk
H huovelin     #jemxadr 0 *real name unknown*   ~huovelin@us.astro.helsinki.fi
H njw          #jemxadr 0 Niels Joergen Westergaard         ~njw@heao1.dsri.dk
H oxborrow     #jemxadr 0 Carol Anne Oxborrow        ~oxborrow@130.226.216.181
H Peter@       #jemxadr 0 KRETSCHMAR Peter         ~pkretsch@isdcul10.unige.ch
315 apollo.dsri.dk * End of /WHO list.      
<njw:#jemxadr> I have nothing to add to the agenda. I believe the finishing time tuesday is dictated by
+ departure times ? END
#jemxadr> Okay, count down to agenda discussion...5...4...3..2..1..Now END
*** nl <~nl@heao1.dsri.dk> joined channel #jemxadr
<njw:#jemxadr> sorry for the head start END
#jemxadr> Hi Niels, good to have you with us. We're just discussing the agenda for the meeting. Anythin
+g you'd like to add? END
<nl:#jemxadr> No, today I am just the fly on the wall
<allan:#jemxadr> I have included a small discussion of the link to ISDC - requirements
<allan:#jemxadr> from the SDAST to the ISDC. THis is an AI on me from ISDC , that 
<allan:#jemxadr> I need to get out of the door.  It will happen around noon monday. END
#jemxadr> Any other additions? END
<huovelin:#jemxadr> By the way, is it very cold there in Copenhagen ? Here we have cryogenic weather EN
+D
<allan:#jemxadr> Currently around -5 C
<allan:#jemxadr> END
<Peter:#jemxadr> No, I just hope we get this annoying raw data formats discussion finished. END
#jemxadr> I'm scheduled to discuss DD, but I really don't have a lot to say: we're doing it. I haven't 
+heard of any particular progress from others. Perhaps during that spot we could each just give a VERY 
+brief account of where we are on DD. END
<allan:#jemxadr> CAO: That was my intention, your name was on there just to 
<allan:#jemxadr> control the discussion... :-) END
<Peter:#jemxadr> Fine with me, except that there's not much to tell. END
#jemxadr> I have things to tell about my work, but not other peoples'. END
<allan:#jemxadr> Peter: Do you think we should extend the telemetry-format discussion? END
#jemxadr> Is it even worth discussing data structures for the next month or so, until ISDC know what th
+ey want? END
<allan:#jemxadr> We need to know in which direction (or lack of?) ISDC is heading END
*** larsson <~larsson@tenma.dsri.dk> joined channel #jemxadr
#jemxadr> Hi Stefan, we're discussing the agenda for the meeting, and specifically data structures and 
+telemetry formats END
<larsson:#jemxadr> Good morning. My train was stuck for 1,5 hrs this morning (its cold!) END
<Peter:#jemxadr> It is worth discussing data structures. Currently I plan to present two clear options 
+and hope to have an _informed_ decision between these two.
<Peter:#jemxadr> The whole current problem essentially arose because SP and me had an idea which _almos
+t_ worked. END
#jemxadr> If there are two possiblities that require our input, then of course, a thorough discussion i
+s required. END
<allan:#jemxadr> Stefan : About the agenda, we would like to know if you have found issues missing. END
<larsson:#jemxadr> Nothing on my mind now. END
<nl:#jemxadr> 1~I am trying to clear up the uncertainties in the Count Rate Format with Grzegorz, Soren
+ and Ib. Hopefully we have it sorted out by monday. END
<Peter:#jemxadr> That would be nice END
#jemxadr> Even if it's not complete sorted out by then, a clarification of how things stand would still
+ be valuable. END
<allan:#jemxadr> Niels:  Tuesday could do -  :-) END
<nl:#jemxadr> Tuesday it will then -what about wednesday?
<allan:#jemxadr> No. We need to finish the issue before we leave (that is before lunch
<allan:#jemxadr> Tuesday END
#jemxadr> So there's nothing except allan's ISDC AI-based issue to add to the agenda?
#jemxadr> and we all know that we have to give a summary of our current DD status? 
#jemxadr> And NL and the other instrument `boys' will regale us with tales of countrate data. Is that a
+ll? END
<allan:#jemxadr> Seems so END
#jemxadr> Okay, on to the discussion of the background handling problem.
#jemxadr> I hope everyone has seen the lovely document sent round by NJW. It sets out the whys and wher
+efores pretty succinctly I think. END
<allan:#jemxadr> Could we please hold this discussion for 5 minutes or so? END
<allan:#jemxadr> I have invited Gerry to participate from 10 (Danish time) and 
<allan:#jemxadr> unfortunally, I have forgotten to include his name
<allan:#jemxadr> in thte list of persons receivingthe instrucitons about how to IRC.
<allan:#jemxadr> He has received them now (I hope), but should be given a few minutes
<allan:#jemxadr> to get the system running (if possible) END.
#jemxadr> Perhaps the most fundamental question, as I a beginner and outsider see it is on page 4, sect
+ion 3.1: what will happen if we don't do anything with the background? Is there ANY way of knowing thi
+s before it's too late? END
#jemxadr> We'll wait five minutes for Jean and Gerry. Any one want to compare low temperatures? END
<allan:#jemxadr> I should add, that JEan has already announced not to be able to
<allan:#jemxadr> join the Chat, too busy with SAX proposals,... END
#jemxadr> Could you get on the phone, Allan, and see if Gerry's actually going to join us? END
<allan:#jemxadr> He is trying ... END
<nl:#jemxadr> While waiting ffor Jerry I can report that the INTEGRAL project has agreed to implement t
+he hardware to eliminate the cross-talk between the two JEM-X units. END
<Peter:#jemxadr> Good. END
<njw:#jemxadr> very good, Niels END
<allan:#jemxadr> vertically or horisontally?
<allan:#jemxadr> END
#jemxadr> Good: any further news on how long the launch might be delayed? End
<nl:#jemxadr> They will implement the horizontal solution - two shelves END
<nl:#jemxadr> No news on launch delay END
<allan:#jemxadr> Except that a delay 'will happen' ! END
#jemxadr> I just got an email from Gerry - he's trying to connect even as we chat, so be patient. END
#jemxadr> How's Oersted today, Niels? Or is that a sore point you'd rather not discuss? END
<nl:#jemxadr> The ORSTED team is relieved that the Delta-II did not go up in flames after the abortewd 
+launch attempt yesterday. Next launch attempt will come in about 10 days. END
#jemxadr> Look on the bright side: everyone's really going to remember this satellite: if it had gone u
+p first time they'd only have remembered it till the next royal faux pas END
<nl:#jemxadr> Question to njw: why did you expect the background counts to have a peak in the center of
+ the detector?
#jemxadr> Are there any non-background matters we should be discussing: how's the coding and DD going? 
+END
<allan:#jemxadr> (Gerry is not answering the phone.  I have even sent instructions about
<huovelin:#jemxadr> What about the Monthly report ? We may not have much to report  END
<allan:#jemxadr> how to log on here and do irc.  he's probably busy installing the SW
<allan:#jemxadr> on a machine not in his office?). END
<Peter:#jemxadr> Coding/DD: I've just started on my first executable, no real DD yet, but some coding w
+ith PIL, RIL, ... END
#jemxadr> About monthly reports: I'll send a new report template to you all next wednesday or so. Numbe
+rs for percent complete on various phases, are apparently expected by ISDC at the very least. END
<huovelin:#jemxadr> We are also tstruggling with PIL, RIL and DAL, but just only
<huovelin:#jemxadr> managed to start succeeding in using them END
#jemxadr> This month you should all at least be able to report so progress with DD/prototyping/coding: 
+let me know how much on the template. END
<njw:#jemxadr> Reply to NL: because the edge of the detector could be more effectively
<njw:#jemxadr> shielded by mask edge and collimator END
<allan:#jemxadr> And we will have to comment on the data structures in the m.r. :-) END
#jemxadr> I've started from the bottom up: I'm leaving all the ISDC interface stuff till the very last 
+bit of coding, so that everyone else can discover the problmes with the ISDC libraries: I don't want t
+o be wasting too much time on these things 
#jemxadr> while there's so much other coding, actuall processing, to be done. END
<njw:#jemxadr> To Juhani and Sami: there are some examples of the use of DAL and PIL in our developer's
+ Forum END
#jemxadr> We'll worry about the actual contents of January's monthly report after the meeting. END
<Peter:#jemxadr> You now, for my first tasks, actual processing is almost zero, so it's natural to star
+t with these interface problems. Also, I'm just learning C en passant. Whoever thought up string handl
+ing in C deserves several eons in unpleasant afterlife ... END
<allan:#jemxadr> Hm - we should be able to write it today - but never mind END
<njw:#jemxadr> BTW Sami, how is the simulation going ? END
<huovelin:#jemxadr> We have already utilized NJW's examples, and also the simulation seems to work END
#jemxadr> Please, if anyone has grappled with the complexities of the DAL and found some actual working
+ code, sent it to me for inclusion on the forum webpage. END
<allan:#jemxadr> ABout Gerry: he claims in his email, that he is trying to install the SW, but I called
+ his office, and they claim he is at another meeting ?  I think we should start the BG discussion when
+ the previous ends, and have a link on the WWW to
<allan:#jemxadr> CA's logging for Gerry to catch up, should he succeed to join END.
<Peter:#jemxadr> Fine with me, let's start. END
<njw:#jemxadr> About section 3.1 (How serious is the problem?) can I report that 3 (not very experience
+d) students
<njw:#jemxadr> will start working on this question and put GEANT/TIMM results in the
#jemxadr> Allan's just sending a message to Gerry with the URL of the chat log, so he can catch up, if 
+he can reach us. END
<njw:#jemxadr> simulation END
<huovelin:#jemxadr> Where do you hidethose results (spatial distribution) ? END
<njw:#jemxadr> The students will actually be present at the BKG discussion part of our meeting monday/t
+uesday END
#jemxadr> That's good, about the students. I'm interested to hear the result of their endeavours. END
<njw:#jemxadr> To Juhani: I don't have them at the moment but Marco Feroci has the result of at least o
+ne run of GEANT. END
#jemxadr> Also, is it possible to produce end products, with and without background corrections, for th
+ose astronomers who simply must do everything for themselves. END
<larsson:#jemxadr> It must be possible for the observer to redo the background corrections. END
<huovelin:#jemxadr> We intend not to do the part of processing automatically , but leavethe applying of
+ background correction to the user END
<Peter:#jemxadr> Is there an onvious way for this facility, given our current architecture? END
#jemxadr> Or leave them out entirely. END
<Peter:#jemxadr> Let's get this sorted out for me: normally image reconstruction includes bakcground su
+btraction in our current scheme - right? 
<huovelin:#jemxadr> I thought not (to Peter) END
<Peter:#jemxadr> If this is true, how do you precisely foresee the user to chose a different or no back
+ground? I have no doubts it is possible, but I'd like a precise description of how and where. END
<njw:#jemxadr> In the image reconstruction by correlation the background is removed 'automatically'
<njw:#jemxadr> under the assumption that it is uniform, which it is not. I think, therefore,
<huovelin:#jemxadr> But I think they mean the instrument (uneven) background (to NJW) END
<njw:#jemxadr> that at least a simple version of the background model should be applied END
#jemxadr> The astronomers I know seem very wary of any data they considered to be `messed about' too mu
+ch, so there must be options to process the data as little as possible, if that's what the end-users w
+ant. END
<njw:#jemxadr> As long as we supply corrected data, not background corrected, and access to
<huovelin:#jemxadr> My impression was that we should provide weights (or probabilities) for each event 
+to a background event, and then someone could just apply that information in his/her analysis END
<njw:#jemxadr> the background libraries, it should be fine for guest observers END
#jemxadr> Yes, the background library access seems like the right way to go. ENd
<larsson:#jemxadr> To Juhanis comment: Those weights will depend on backgr. model END
<Peter:#jemxadr> Yes, but one of the problems is, that cross-correlation will always remove some backgr
+ound and it might be wiser to try and remove 'all' when deriving those images and source data in autom
+ated analysis - or even interactive, as long as the user choses to run a 'standard' pipeline. END
<huovelin:#jemxadr> to NJW: That maybe be too dangerous, since the user (guest observer) may not have e
+nough experience on how to apply different ways of determining the background) END
<nl:#jemxadr> I think we may include some simple background terms (simple radial varying terms or unifo
+rmly tilted planes in the correlation analysis - this may be good enough to allow source identificatio
+n - detailed analysis must be left to the user. E
#jemxadr> Rolling the background corrections in with image reconstruction seems like a one-way process 
+which cannot be removed. END
<Peter:#jemxadr> To JH: Whatever we think of the GO's knowledge, the possibility must be there too redo
+ all. END
<huovelin:#jemxadr> Yes. That's right. I agree also with CAO END
<njw:#jemxadr> To CAO: I don't agree, image reconstruction and source finding can always be repeated EN
+D
<nl:#jemxadr> The "serious" user will probably always start from pretty raw data END
<allan:#jemxadr> He/she will, but we do have obligations to provide
<allan:#jemxadr> something in the SA (and QLA). END
<huovelin:#jemxadr> Yes, but there could be different levels of "correctly" reduced data END
<Peter:#jemxadr> To NJW: my memory is slightly hazy, but wasn't there a refinement of cross-correlation
+ that did not lead to average, smooth part = 0? END
#jemxadr> Yes, you must expect the observer to have a certain amouunt of skill, commensurate with the a
+mbitiousness of their project. If they want to do a `standard' analysis, directed by experts (us) then
+ they'll get one sort of data. If they want t
#jemxadr> to be more independant, then they're going to have to have a little sophisticated knowledge a
+bout backgrounds etc. It's a reasonable expectation from our side, I think. END
<allan:#jemxadr> I don' t think they can go ahead and publish SA data ,  I wouldnt dare, at
<allan:#jemxadr> least END
<njw:#jemxadr> To Peter: there are several options with cross correlation. One is to have the smooth di
+stribution disappear END
<njw:#jemxadr> I agree to Allan's statement: the SA results are just guidelines END
<nl:#jemxadr> If we talk about fitting the data rather than cross-correlating the issue of handling the
+ background becomes more explicit - if not easier. END
<allan:#jemxadr> I am sorry to interrupt a nice discussion, but could we back up and do a minor agenda 
+for what we need to discuss today and what we rather should do Monday? END
<allan:#jemxadr> I suggest to have now (since Gerry is not here) questions directly 
<allan:#jemxadr> related to the interpretation of NJW's docuiemnt. END
#jemxadr> I think Monday should be devoted to the actual dynamics of background determination, while to
+day we talk about more general issues: how much BKG etc.
#jemxadr> How will we use background in our syste? What do users expect background-wise? END
<allan:#jemxadr> - if we need to go into  these details, for how long will we extend this meeting END
<huovelin:#jemxadr> Until we have answered all questions we should on Monday/Tuesday (to Allan) END
<Peter:#jemxadr> I'm not sure we can deal with all the topics CAO raises today.
#jemxadr> As for the document itself, it seems very complete in setting out the problems and the list o
+f things to be answered - and a good place to start an involved discussion. END
#jemxadr> Strictly from the document point of view, I don't thinks there's anything to discuss today. E
+ND
<huovelin:#jemxadr> I agree . Maybe we should stick to a short meeting today END
<allan:#jemxadr> My impression too END
<Peter:#jemxadr> A lot of this seems to be rather fundamental and linked with the various mental pictur
+es of how the software works and will be used. END
#jemxadr> Since Gerry isn't with us, shall we all call it a day? END
<njw:#jemxadr> In that case I'll call for specific comments to the document - is there any ?? END
<Peter:#jemxadr> One moment please, I have one more point. END
*** guest <~guest@tenma.dsri.dk> joined channel #jemxadr
#jemxadr> Hi `guest', please identify yourself. END
<Peter:#jemxadr> NJW, SL and me should summarize how background is supposed to be taken into account wi
+thin our tasks of the ISSW, i.e. ImageGeneration, SourceDataExtraction and DataBinning. This summary s
+hould point out who this treatment may differ between QLA, SA and offline analysis. END
<guest:#jemxadr> msg
<Peter:#jemxadr> While the ADD says a lot, it seems to me this information isn't fully "on the table" f
+or everyone involved. END
<allan:#jemxadr> I  guess our guest is Gerry Skinner, right?
<guest:#jemxadr> names
<allan:#jemxadr> END
<guest:#jemxadr> msg
<guest:#jemxadr> help
<guest:#jemxadr> help msg
<allan:#jemxadr> Gerry i f you hit /who you'll see who's on.
<allan:#jemxadr> Further, just type ahead, and we all see what you write. END
<allan:#jemxadr> (We end sentences with END ) END
#jemxadr> Hello Gerry, it's good to have you with us. If it is you. END
<guest:#jemxadr> Guest is Gerry Skinner , but I have no idea how this system works
<huovelin:#jemxadr> It works fine, as you see END
<nl:#jemxadr> sorry, I have to leave you now - see you on monday
<huovelin:#jemxadr> Welcome  ! END
<Peter:#jemxadr> As long as you just type text, everybody can read it. END
#jemxadr> I'm logging the chat session, so you can catch up with us if you want at the address allan se
+nt. END
*** Signoff: nl ()
<guest:#jemxadr> OK, I genin to get the idea END
<allan:#jemxadr> The URL is sent as email, did you get that?
<allan:#jemxadr> END
<njw:#jemxadr> To Peters comment: I fully agree, we obviously need to clarify this END
<larsson:#jemxadr> NJ/Peter: Agree. END
<huovelin:#jemxadr> Clarify what ? END
#jemxadr> Gerry, could you give us any comments you have on NJW's discussion document, since we've deci
+ded not to discuss the technicalities of background determination itself. END
<guest:#jemxadr> Having made contact, I think it is probablyt best if I just wait to review the transcr
+ipt after wathing any further transactions - unless there are specific issues/questions for me. END
#jemxadr> Specfic issue: what do you think of the discussion document: anything missing? Anything that 
+can be discarded immediately? etc. END
<guest:#jemxadr> My comments on the discussion document are that it is a very good start, and the most 
+of what I could add is very detailed. Perhaps the main thought I had was that filtering in the detecto
+r plane is largely equivalent to filtering the image
<guest:#jemxadr> END
#jemxadr> I think we should definitely leave the details to monday. But we're all satified with the doc
+ument: many thanks to Juhani, Niels Joergen and Stefan. END
<huovelin:#jemxadr> OK. It is much easier to be around the same table and ask the questions then END
#jemxadr> Any comments on detector plane vs. image filtering? Does this affect the document now in any 
+way? END
<huovelin:#jemxadr> I think that's a kind of "detail", which cN BE LEFT TO MONDAY END
#jemxadr> We maybe don't have to consider these as two separate issues? END
#jemxadr> Well, then, now that the agenda is agreed, and the document approved for detailed discussion,
+ I think we should finish for today.
<guest:#jemxadr> It was just that the document seemed to assume that filtering was done first, before i
+magein END
#jemxadr> apologies to Gerry, who only just joined us. In future we will host somewhat more regular bac
+kground chats, and I hope that the knowledge and setting up you've done today will be of help to you i
+n the future. END
<Peter:#jemxadr> Wait a second, CAO.
<njw:#jemxadr> Gerry, could you elaborate a bit on the term filtering: does it mean smoothing on a spec
+ific scale ? END
<Peter:#jemxadr> To Gerry: the instrument on-board hard- and software will filter out a large fraction 
+of the particle bkg, at least this is expected. Do you refer to this? END
<guest:#jemxadr> Yes, I use the term to refer to correcting slowly varying levels, either by filterimg 
+out low frequencies or by subtracting a polynomial or spline or whatever END
<guest:#jemxadr> Sorry, the YES was to the comment about smoothing on a particular scale. I did not mea
+n YES to Peters conclusion that I was talking about background rejection such as done on board.
<guest:#jemxadr> END
<huovelin:#jemxadr> to Gerry: But wouldnt it be different in the two frames of reference, since the ins
+trument background is not dominated by mask coding END
<Peter:#jemxadr> No problem, it was very clear from the rest of the sentence. END
<allan:#jemxadr> Are we then dressed to continue Monday? END
#jemxadr> I think that it's important that we all begin on monday with the same terms of reference.
#jemxadr> Therefore, could I suggest that we start the discussion with Juhani presenting a little on th
+e background problem and THEN Gerry do the same, so that we can found out the different ways we each d
+escribe the same things. eND
<njw:#jemxadr> good suggestion END
<larsson:#jemxadr> Gerry, If you have any additional points or questions to us regarding these
<larsson:#jemxadr> issues you could maybe send us a mail during the day (will give us something
<huovelin:#jemxadr> OK. I'l try to tell you, what's bugging me in the background END
<larsson:#jemxadr> to think about during the weekend)? END
#jemxadr> We don't want to find out after a couple hour's discussiont that we've been talking at crosse
+d purposes. END
<guest:#jemxadr> I am logging off now. I will look at the transcript and if there is anything else
<guest:#jemxadr> we can exvhange e-mail or leave it to MondayBYE
#jemxadr> Allan, could you puton the more detailed agenda 15 mins each fro JH and GS to present their c
+ases? END
#jemxadr> Bye Gerry, thanks for joining us. See you on Monday. END
<allan:#jemxadr> I'll do that. Agenda comes out right after this END
<allan:#jemxadr> See you Monday, have a safe trip! END
<njw:#jemxadr> See you Monday - have a nice day and then week-end BYE
*** Signoff: njw ()
<Peter:#jemxadr> See you all on Monday - looking forward to it! END
#jemxadr> See you all on Monday. Have a good weekend. END
<huovelin:#jemxadr> Bye bye, see you monday END
*** Signoff: huovelin ()
<allan:#jemxadr> quit
<larsson:#jemxadr> See you all on Monday. And to CA: Mineral water is great! END
#jemxadr> Bye!
*** Error: Closing Link: oxborrow[~oxborrow@130.226.216.181] () 
*** IRC session log ended at Fri Jan 29 11:18:48 1999.
