 Weekly chat meeting # 24: Friday 19th November 1999

Participants: Carol Anne Oxborrow, Peter Kretschmar, Allan Hornstrup, Niels
	Joergen Westergaard, Sami Maisala, Eero Esko, Stefan Larsson

Subjects discussed: AI list:
	Contributions to ADD5.0 and DDD1.0
	Test telemetry from refurbished EM
	Raw data verification program
	tkirc
	data structures: names, consistency, dedicated chat, images
	background data: storage format
	rejected events
	background retrieving executable
	non-telemetry data storage
	calibration chat
	grey filter: obt of autonomous events

Action Items:

AI991119_1	CAO	ASAP	Talk to IT about getting test telemetry
packages sent to ISDC
OPEN

AI991119_2	SM/EE 	ASAP	Distribute tkirc to SDAST so that it can be
tested by the whole team in December
OPEN

AI991119_3	AH	Mid-Dec Get tkirc running at DSRI in time for test
chat-meeting with this software.
OPEN

AI991119_4	ALL	3/12/99 Prepare lists of I and O data structures needed 
for each component in time for data structures chat on Friday 3rd dec.
OPEN

AI991119_5	PK	3/12/99 Prepare summary of ISDC policy on the naming
of image data structures, from SPI data structure proposals.
OPEN


*** IRC session log started at Fri Nov 19 09:27:20 1999.
H oxborrow@    #jemxadr 0 Carol Anne Oxborrow          ~oxborrow@tenma.dsri.dk
315 apollo.dsri.dk * End of /WHO list.      
*** Peter <~pkretsch@isdcul10.unige.ch> joined channel #jemxadr
#jemxadr> Hi Peter, how are you? I have a fridge full of herring which should h
+ave
#jemxadr> been delivered to you yesterday! END
<Peter:#jemxadr> Good morning Carol Anne! Thanks, I'm fine - things are just a 
+little frantic around here right now.
<Peter:#jemxadr> I'm looking forward to the Sild delivery next week. When will 
+you arrive on Tuhrusday evening, btw? END
#jemxadr> Frankly, I didn't know it was possible to be so busy and not drop dea
+d ......but now I know better! END
#jemxadr> We arrive on friday and leave on friday, so it's something of a surgi
+cal strike. Be prepared! END
#jemxadr> I'm deeply embedded in making mini-preprocessing to verify the test d
+ata that's coming out of the refurbished EM.
<Peter:#jemxadr> Allan and Soren wrote, they'd arrive on Thursday - is that no 
+longer true? We do especially need some time with Soren END
#jemxadr> She, JEM-X is sitting downstair pumping out bits that make Soeren and
+ Rene
#jemxadr> scratch their head and wander around in a daze! END
#jemxadr> I think Soeren's going to arrive a little earlier. Let me double chec
+k my arrangements. END
#jemxadr> I'll just go an rustle up the other boys. END
<Peter:#jemxadr> Good idea! END
*** allan <~allan@apollo.dsri.dk> joined channel #jemxadr
<Peter:#jemxadr> Hi Allan! CAO just told me that you would arrive on Friday. Wh
+en exactly? END
*** smaisala <^smaisala@mizar.astro.helsinki.fi> joined channel #jemxadr
*** eesko <~eesko@aquarius.astro.helsinki.fi> joined channel #jemxadr
#jemxadr> Hi, Allan, sami and eero, how are you all? NJW will be with us soonEN
+D
*** njw <~njw@heao1.dsri.dk> joined channel #jemxadr
<njw:#jemxadr> good morning END
<eesko:#jemxadr> Hello all. END
<allan:#jemxadr> to PK: Same as CAO : 10.45 at the airport, if all goes well. E
+ND
<allan:#jemxadr> (or is it 10.50 END).
#jemxadr> Let's wait a couple of  minutes for stefan, then begin ENd
#jemxadr> Any pre-meeting business? END
<Peter:#jemxadr> To CAO, AH: so this means we can begin this discussion essenti
+ally over lunch - right? END
#jemxadr> That depends - what will there be for lunch ;-) ? END
<Peter:#jemxadr> For the others: sorry for any confusion - AH, SB and CAO will 
+be at ISDC next week to dicuss the future of QLA ... END
#jemxadr>  Of course we want to get the discussion started ASAP once we arrive.
+ END
<Peter:#jemxadr> If you want I could organize lunch for the two or three (depen
+ding on Soren's choice to buy soemthing himself) of you. END
#jemxadr> Okay, it's 10 minutes over the official meeting time, so let's begin 
+without Stefan. I want to run through the AI list as quickly as possible, and 
+we can discuss anything 
#jemxadr> that arises END
<Peter:#jemxadr> Fine with me - get going. END
#jemxadr> Peter: I think we'd be very happy if you could arrange sandwiches for
+ next friday - something quick. END
#jemxadr> Okay, on the first to AIs: we are way behind our originally intended 
+schedule with ADD5.0 and ddd1.0. Lets forget DDD1.0 till
*** larsson <~larsson@tenma.dsri.dk> joined channel #jemxadr
#jemxadr> more of V2 is written, and delivered. On ADD, you need to send update
+s
#jemxadr> for background, templates for any new data structures and short descr
+iptions of each component to go in the written overview. END
#jemxadr> Hi Stefan, we're going over the action item list. END
<larsson:#jemxadr> Hi CA and others, sorry to late. END
#jemxadr> So let's update the due dates on AI990802_3 and _4. We'll make the AD
+D
#jemxadr> updates due when? Mid Jan might be most realistic - so we
#jemxadr> ve got time to finish background documents and technical notes. What
#jemxadr> do you all think? END
<Peter:#jemxadr> Sounds reasonable to me. The only question is - do we make the
<Peter:#jemxadr> due date before our planned meeting in January or one week aft
+er that
<Peter:#jemxadr> for final touches? END
#jemxadr> I think a week after is better than before the meeting END
<smaisala:#jemxadr> I agree END
#jemxadr> Okay, we make that one due mid Jan sometime to be specified more clos
+ely later. END
#jemxadr> I think we'll wait till after v2 delivery deadline for DDD - so
#jemxadr> that one becomes mid-April. Is that okay with you all? Is that realis
+tic? END
<njw:#jemxadr> Y
<Peter:#jemxadr> Probably Y
<smaisala:#jemxadr> I hope so Y
<larsson:#jemxadr> Y too. 
<eesko:#jemxadr> Y
#jemxadr> AI990903_1 : this one is closed. Test data is pouring out of the EM
#jemxadr> downstairs, and I am in the process of putting it into ISDC data stru
+ctures. The repository of test data in the proper data structures should be av
+ailable withint the  next week END
<Peter:#jemxadr> ISDC would probably like to get hold of some 'real' TM
<Peter:#jemxadr> in order to test its preprocessing. Do you think this could be
+ organized?
<Peter:#jemxadr> Files with N full packets in them would do as baseline. END
#jemxadr> As regards the actuall telemetry, you should speak to Soeren aobut th
+at - I'm not using raw telemetry but sorted telemetry that Soeren produces and
+ passes to me. END
#jemxadr> I'll talk to Soeren about getting the current telemetry sent to ISDC 
+ASAP. It could be that they don''t feel that what's
#jemxadr> being produced at the moment is so useful inasmuch as there's several
#jemxadr> SW bugs that have to be fixed when Ib gets back from spain. END
<Peter:#jemxadr> OK. We'll take this discussion up with Soeren on Friday mornin
+g. END
#jemxadr> Okay that's a new AI on me to get the telemetry packets sent if possi
+ble. Now about the checklist for data sanity:
#jemxadr> I'm embroiled in our mini-Preprocessing here, and learning about tele
+metry--->ISDC connection bit-by-bit. However, a check list of 
#jemxadr> `good' data properties is slowly emerging. I have thought of making a
+ 
#jemxadr> program to test for all these things in a give set of data - wouldyou
+ guys like to have such a program? END
<Peter:#jemxadr> Depends on how much time it is for you to write it.
<Peter:#jemxadr> And how far reaching would this check be? END
#jemxadr> I think I'm going to have to  do it anyway as part of our telemetry v
+erification program here - it would include all the checks that I've already d
+iscover need doing and any that Soeren or
#jemxadr> anyone else cna think up: last event in a packet comes before the fir
+st event in the next packet etc. Last 7 bits of OBT are
#jemxadr> zero etc. END
<Peter:#jemxadr> Sounds quite useful - maybe you should DAL-ify this and includ
+e it in the ScW pipeline :-) END
<Peter:#jemxadr> Anybody else has an opinion? END
#jemxadr>  People are welcome to suggest extra checks of their own, though it's
+ doubtful that anyone who hasn't gone through the telemetery-->ISDC conversion
+ thoroughly could come up with anything extra. It's already dalified - how els
+e can I
#jemxadr> squash things into ISDC formats? ;-) END
<Peter:#jemxadr> The questions rather what you read and how ... END
#jemxadr> Okay, we'll change AI12.3 to a later date, say mid-Jan 2000, and exte
+nd it to include a basic program for checking input data END
#jemxadr> If the program really proves useful we can talk about making part of 
+the ScW pipeline JVerify, or something like that END
<Peter:#jemxadr> Fine with me END
#jemxadr> Next is AI12.4 on Sami - have you found something better than IRC, Sa
+mi? END
#jemxadr> Sami?? END
<smaisala:#jemxadr>  IRC is OK. But we (Me and Eero) are testing today new irc 
+program and it seems to be much more better than the old one END
#jemxadr> What's it called  and where can we all get it? END
<smaisala:#jemxadr> The name is tkirc and It is graphical interface to ircii pr
+ogram END
#jemxadr> Is it quicker than current IRC? I think speed is the principal proble
+m at the moment, now a lack of graphics. END
#jemxadr> NOT a lack of graphics END
<Peter:#jemxadr> Or more generally: What do you prefer about tkirc? END
<allan:#jemxadr> WhaT is wrong with the speed?
<allan:#jemxadr> The problkem is how fast you write, isnt it? END
#jemxadr> Doesn't it also take quite a time to transmit? It seems very slow to 
+me END
<allan:#jemxadr> I dont think so ... ? END
<eesko:#jemxadr> It's just Sami, the program is quite fast
#jemxadr> Perhaps we all just need to learn to type faster, and not wander off 
+to the toilets during a meeting...END
<eesko:#jemxadr> END
<eesko:#jemxadr> That's it END
<smaisala:#jemxadr> I think the problem is in connections if it is slow. END
<smaisala:#jemxadr> I agree CAO. It is up to us END
<Peter:#jemxadr> Back to my question: what _is_ better in tkirc than in what we
+ use now? END
#jemxadr> Could you tell us where to get this software, and we can test it out 
+some time? END
<smaisala:#jemxadr> We can send it to you by email END
<smaisala:#jemxadr> It is easy to install and use END
<eesko:#jemxadr> You can scale the screen :) END
#jemxadr> Allan, could you take charge of setting up this new software here so 
+that we can try it out? END
<eesko:#jemxadr> Without restarting the program END
#jemxadr> I like the sound of scalling the screen.....then I can read the actio
+n list too END
<smaisala:#jemxadr> You can scroll the screen :-) END
<eesko:#jemxadr> You can also scroll to see what has been said earlier END
#jemxadr> All that sounds better. Let's try it. ENd
#jemxadr> AI on sami and eero to distribute the tkirc software. AI on Allan to
<smaisala:#jemxadr> First you have to check that you system has tcl/tk version 
+8.0 installed, but it is easy to get. END
#jemxadr> install it a DSRI???? END
<Peter:#jemxadr> OK, so we'll all try it (hoping I get someone here to install 
+it) and report. END
<Peter:#jemxadr> Next topic? END
#jemxadr> I think the AI on reading DAL3JEMX manual and forwarding bugs to Step
+hane should be moved to `ongoing', since we're going to be
<smaisala:#jemxadr> To Peter: It is really easy to install and you can put it i
+n your own home directory :-) END
#jemxadr> doing this as long as we use the manual. I have a could of bugs to re
+port, but haven't got round to emailing him yet. END
#jemxadr> AI12.13 has to do with checking consistency of data structures: I had
<Peter:#jemxadr> Yes, but please be aware that the earlier he gets this info, t
+he higher the chance he will actually correct it before the delivery dat of yo
+ur iSSW. END
#jemxadr> originally thought to do this using ADD5.0, but to get a start now we
#jemxadr> need to interact with eachother directly.
#jemxadr> So I propose that we each look at what data structures we use as inpu
+t and produce as output.
#jemxadr> Then we each contact the person who creates our input structures and
#jemxadr> discuss with them the exact format/content of the data structure. We 
+also
#jemxadr> contact the people who'll be using our output structures and do the s
+ame. Can we do this? END
<Peter:#jemxadr> IMHO yes. END
#jemxadr> We need to be sure that when we each use a particular name we're all 
+clear about what that name means and what the data structure it describes actu
+ally contains. END
#jemxadr> Okay, let's set up a procedure: first we each need to make a list of 
+our input structures to each of our components and a list of output structures
+. Let's say that these lists should be ready next week.
#jemxadr> Can we have a data structures-only chat next week? END
<Peter:#jemxadr> Small problem: next week is _completely_ filled with meetings 
+for me.
<Peter:#jemxadr> And rememebr next Friday you are here. END
<smaisala:#jemxadr> So maybe we do it week later ? END
#jemxadr> Of course, you're right Peter! What about a dedicated data structure 
+meeting on Friday 3rd or Friday 10th? END
<Peter:#jemxadr> Friday 3rd sounds like a good idea. END
<larsson:#jemxadr> OK with me. END
#jemxadr> Okay that's an AI on EVERYONE to have a list of I and O data structur
+e
#jemxadr> ready for Friday 3rd chat meeting dedicated to data structures. Okay?
+ END
<Peter:#jemxadr> OK NED
<larsson:#jemxadr> Okay. END
<smaisala:#jemxadr> OK END
<eesko:#jemxadr> OK END
#jemxadr> This meeting could also be used to close AI12a.1 on NJW since that is
#jemxadr> about data structures. Has anything already been finalized about imag
+e
#jemxadr> data structures yet? END
#jemxadr> NJW?? END
<Peter:#jemxadr> There is a rather detailed proposal from ISDC, currntly probab
+ly "hidden" within the SPI data structures. 
<Peter:#jemxadr> I will see if i can write up a summary later today. END
<njw:#jemxadr> Yes, alright END
#jemxadr> Okay that's an AI on Peter to write a summary on ISDC policy on image
#jemxadr> data structures. END
<Peter:#jemxadr> Due by Dec 3, just to play safe ... END
#jemxadr> Next AI is on NJW to find out if there's a change in number of bits i
+n the TIME format. We'll leave that till end of november, and also AI12a.11. C
+urrently due however is AI12a.13
#jemxadr> on SL to make a technical note on background data storage. Do you wan
+t
#jemxadr> and extension on that one stefan? END
#jemxadr> Will your email of today be an official document I can put on the web
+ any
#jemxadr> time soon?
#jemxadr> END
<Peter:#jemxadr> We must be careful there - this needs to be fixed relatively s
+oon. END
<larsson:#jemxadr> You all got the preliminary note this morning. Any comments 
+on that? ?
#jemxadr> If our background tzar can approve your email, perhaps it can quickly
+ be
#jemxadr> turned into a nice tex document. Juhani? END
#jemxadr> I have no comments on your email stefan, but then background isn't re
+ally
#jemxadr> my problem: telemetry is! END
<Peter:#jemxadr> The question I have is if everybody agrees that this is the ac
+tual interface
<Peter:#jemxadr> for the analysis software. I.e. if the software from Helsinki 
+produces this 6 datastructures for us and we use them as described.
<Peter:#jemxadr> Is everybody fine with that?
<larsson:#jemxadr> I am. END
#jemxadr> Yes END
<Peter:#jemxadr> One other question - has anybody thought about the names? Do y
+ou prefer Sami's scheme or mine? END
H larsson      #jemxadr 0 Stefan Larsson                ~larsson@tenma.dsri.dk
H njw          #jemxadr 0 Niels Joergen Westergaard         ~njw@heao1.dsri.dk
H eesko        #jemxadr 0 Eero Esko          ~eesko@aquarius.astro.helsinki.fi
H smaisala     #jemxadr 0 Sami Maisala       ^smaisala@mizar.astro.helsinki.fi
H allan        #jemxadr 0 Allan Hornstrup                ~allan@apollo.dsri.dk
H Peter        #jemxadr 0 KRETSCHMAR Peter         ~pkretsch@isdcul10.unige.ch
H oxborrow@    #jemxadr 0 Carol Anne Oxborrow          ~oxborrow@tenma.dsri.dk
315 apollo.dsri.dk * End of /WHO list.      
<smaisala:#jemxadr> I Think the Peter's names are good. I agree them. END
#jemxadr> Okay, let's go with Peter's names. END
#jemxadr> Well, I hear no discontent. could you go ahead please Stefan and turn
+ 
#jemxadr> the email into a official document? END
<larsson:#jemxadr> I will CA. END
#jemxadr> I'll make that due on 3rd Dec too. END
<larsson:#jemxadr> Fine. END
#jemxadr> AI12a.14 is I think closed since Peter sent us his email on data stru
+cture names. END
<Peter:#jemxadr> Final comment on this: Let's hope the intsrumental background 
+does not vary much from position to position. Otherwise we may end up again wi
+t the 256*256*256 cube :-) END
<smaisala:#jemxadr> Peter, You wanted separate sets of   instrument and diffuse
+  spectral data tables. I think it's fine. How about others ? END
<larsson:#jemxadr> Agree! END
<Peter:#jemxadr> Yes, you are right. And if we use your scheme, we could always
+ just have a single line in the diffuse part if that doesn not vary after all.
+ END
#jemxadr> AI12a.16 NJW: are you with us NJW? Do we have the number of rejected 
+events in the HK data? END
#jemxadr> No response there so we'll skip over all the AI's due in December.
<larsson:#jemxadr> Here is the reply I got fro Ib earlier:
<larsson:#jemxadr> The rate of rejected events is stored in HK TM Packet. This 
+appears every 8 seconds. Actually we
<larsson:#jemxadr>      store the number of triggers, accepted events and rejec
+ted events divided according to rejection
<larsson:#jemxadr>      criteria. For some purposes an event rejected by grey f
+ilter might be a good event. 
<njw:#jemxadr> Yes, the number of rejected events is included in the HK END
<Peter:#jemxadr> Sorry for interfering: one thought - if we have a consistent s
+cheme we could write a DAL3JEMX function to calculate the bakcground as functi
+on of (X,Y,E,T). Just an idea. This function would need to be written by one o
+f us, I guess. END
#jemxadr> Okay, that's good work guys, so AI12a.16 is closed. END
#jemxadr> That's a very fine idea Peter - just so long as I don't have to code 
+it - I'm up to capacity very definitley, now. END
<smaisala:#jemxadr> CAO you never know ....END
#jemxadr> How about we save that idea for discussion when Juhani is with us? EN
+D
#jemxadr> AI991112_1 is closed after Juhani sent me an email and
<Peter:#jemxadr> Fine with me. It just flashed through my mind and I thought: b
+etter write it up now, while I still remember. END
#jemxadr> as for AI991112_2 on me: I will be able to set this up by the due dat
+e. END
#jemxadr> AI991112_3 is our last `old' AI, and it is closed since I sent you al
+l (twice) the consolidated AI list. I'll be sure to send
#jemxadr> you guys the updated AI list after every meeting from now on. END
#jemxadr> Do any of you have AOB? END
<Peter:#jemxadr> I have something
<Peter:#jemxadr> This also will take a little time, 
<Peter:#jemxadr> but looking at the work Pierre has done for SPI and 
<Peter:#jemxadr> critically reading our ADD it is apparent that we definitely
<Peter:#jemxadr> need to better define access to all the data that is not deriv
+ed
<Peter:#jemxadr> from telemetry. This means: calibration data, instrument model
+ data,
<Peter:#jemxadr> Good Time Intervals, ...
<Peter:#jemxadr> In many case we mention what we want, but it is not clear if
<Peter:#jemxadr> these structures form part of a Science Window Group ('telemet
+ry'
<Peter:#jemxadr> and auxiliary data will), if they are bundled together in some
+ 
<Peter:#jemxadr> other group (ISDC likes NJW's idea of having a group to bundle
<Peter:#jemxadr> the JMXi-****-MOD data structures), or if we need some
<Peter:#jemxadr> sort of Index table (probably the case for calibration
<Peter:#jemxadr> data).
<Peter:#jemxadr> Probably this goes together with the action we all received
<Peter:#jemxadr> today on organizing our in- and outputs. My whole point
<Peter:#jemxadr> is that in many cases we need to think beyond the
<Peter:#jemxadr> "what is in this data structure" and also ask ourselves
<Peter:#jemxadr> "where do we find/store it". 
<Peter:#jemxadr> END-OF-MONOLOG
#jemxadr> I agree entirely Peter: it's a problem I've already run across, codin
+g
<allan:#jemxadr> I agree with you, Peter, and the discussion is sometimes spora
+dically
#jemxadr> JGainCaclulation. ENd
<allan:#jemxadr> coming up here at DSRI, --- we should soon get together and 
<allan:#jemxadr> do a more clever thinking here.
<allan:#jemxadr> Could that be a major part of the Jan meeting, or is it too la
+te
<allan:#jemxadr> for yuou ? END
<Peter:#jemxadr> I guess it would make a good topic for January, but we really 
+have to work up to that meeting.
<Peter:#jemxadr> And be ready to adapt our code after that meeting, before deli
+very.
#jemxadr> I think it should be discussed in January - this won't hold up coding
+ at
#jemxadr> this point. END
<Peter:#jemxadr> Especially the Calibration data storage needs also more thinki
+ng
<Peter:#jemxadr> within ISDC.
<Peter:#jemxadr> END
#jemxadr> Juhani has also requested that we have a calibration-only chat and
#jemxadr> possibly an in-person meeting devoted to calibration just as we've
#jemxadr> done with the background issue. Could we decide on a Friday for
<Peter:#jemxadr> There is a certain effect on coding, since you retrieve your d
+ata somehwat differently depending on the method (group of various things, ind
+ex of same things, direct access) used END
#jemxadr> a  calibration chat as a preliminary to the January meeting? END
<Peter:#jemxadr> Problem is: Friday 10, I may have OMC people here. END
<Peter:#jemxadr> Are you all still working on Friday, Dec 17 - or wil you be sk
+iing by then? END
#jemxadr> I can make Friday 17th but not 10th. END
<allan:#jemxadr> One of the fridays may be lost to a X-mas party, we dont know
<allan:#jemxadr> which yet, though :-) END
#jemxadr> No it's been decided: christmas lunch will be a thursday. eND
#jemxadr> Let's tentatively say that Friday 17th december is calibration-chat, 
+is
#jemxadr> that okay with the rest of you? END
#jemxadr> Friday 10th will have to be a quick one for going over the AI's END
#jemxadr> I also have some AOB: how do we want Grey event times defined?
#jemxadr> I'm soliciting everyone's opinion on this one. With the information
#jemxadr> available I can calculate the OBT of the events immediately before an
+d
#jemxadr> after the grey filter change (autonomous). So which time is the grey
#jemxadr> event time: the event before, the event after, average of the two?
#jemxadr> Just after the last event? Just before the next real event? Please le
+t
#jemxadr> me know your feelings on this one within the next couple of weeks. eN
+D
<larsson:#jemxadr> I'll think about it. END
<Peter:#jemxadr> My vote is for average. END
#jemxadr> Thanks, Stefan. Please think about it everyone. Any more AOB? end
<Peter:#jemxadr> Not from me. END
#jemxadr> What about you Finns? You're awfully quiet both of you! END
<smaisala:#jemxadr> no END
<eesko:#jemxadr> no END
<larsson:#jemxadr> Nothing from me. END
<allan:#jemxadr> No END
<Peter:#jemxadr> To Eero - thanks for the tkirc mail. END
<smaisala:#jemxadr> You talk so fast that we can only listen :-) END
<allan:#jemxadr> And we are currently installaing at DSRI - closing the AI as w
+e 'speak' END
#jemxadr> Well, that was very productive! Great stuff guys! There will be no ch
+at next week since some of us will be at ISDC discussing
#jemxadr> QLA. `See' you all on Friday 3rd, complete with your data structure l
+ists END
<smaisala:#jemxadr> OK. Have a relaxing weekend END
*** Signoff: allan ()
<larsson:#jemxadr> Bye. END
<smaisala:#jemxadr> \exit
*** Signoff: larsson ()
#jemxadr> A sunny and relaxing weekend to all of you. END
*** Error: Closing Link: oxborrow[~oxborrow@tenma.dsri.dk] () 
*** IRC session log ended at Fri Nov 19 11:23:51 1999.
