Data Processing Tool

Scritto da Nicolas Chauvin on 13 Novembre 2012.

Since the measurement data is quite large every day as the setup is acquiring about 30 full lines measurement per minute, we have developed the small tool in .NET (C#) to compact the data by rewriting a new CSV file with only one measurement line per minute.

All data for each column belonging to the same minute is averaged to generate a single value per minute.

You can download the tool here: MFMP_CSV.zip df
And the C# source code is here: MainForm.cs
(not super clean code, since we changed the data format several times)

For the sake of transparency, we will continue to post raw measurement data.
But is you prefer to work on smaller data set, to make it more manageable to generate graphs in Excel or in any other spreadsheet software, then this tool is good for you.

The software runs on Windows or on a Windows virtual machine (tested on OS X).

< Prec Succ >

Add comment

Comments

0 #18 Robert Greenyer 2012-11-14 02:16

@ Sanjeev

The wire is loading with Hydrogen at T 150 causing resistance to drop.

The resistance going up is evidence of the effect ie the H is being used or loss of H from the wire.

0 #17 Sanjeev 2012-11-14 01:14

The resistance is rising steadily, which may mean that the reactor is not operating at optimal temperature. I guess T_Mica = 150 C would be a good try since the resistance shows favorable behavior in that range.
Perhaps this can be tried after a day or two and after some more loading-reloadi ng cycles.

0 #16 Robert Greenyer 2012-11-13 23:02

@ David

We are going to be crunching numbers and taking a view on the excess calculation metric. Thanks again for your input - it feels like we have you and others on the team and that gives me a warm fuzzy.

With regard to the success so far - we re-created some of Celani's findings, saw interesting things and showed how important energy loss is to the system. Frankly - if, after successful loading, we had turned on the reactor and produced excess heat, I am not sure we would have learnt much and it is a pretty boring story - well.. kinda!

This is far more interesting. Thanks for your support.

@Andrew

It would be totally awesome if you could build a cross-platform tool like that and we could host it on the MFMP site - would be great. Please also allow us to open source the code for others that may add value. Thanks!

@Nicolas

68 Gb in 24 hours!!- with nearly no one knowing about this! I think we might need to consider Torrents for 12 hour chunks of raw data with previous data to the current 12 hours at a smoothed data rate with much lower granularity.

0 #15 Ecco 2012-11-13 22:03

@Nicolas Chauvin: just to state this again, there's much room for bandwidth usage optimization in the live data page, considering that most people probably only want to see data plotted on the website, preferably in real time.

0 #14 Nicolas Chauvin 2012-11-13 21:51

I added the link to the source code in the article for those interested.

Moreover, for people having problem to download CSV file from the live graph HUGnetView page, we currently set a limitation due to limited bandwidth.
You can download 4h of data, but not 12h.

For your information, 68 Gb of data was downloaded in less than 24 hours - this is a HUGE amount of bandwidth at a time when hardly anyone knows about this.

+2 #13 David Jones 2012-11-13 20:34

And I forgot to add a 'thank you' to the MFMP people

Thank you

0 #12 David Jones 2012-11-13 20:28

I think this is a fascinating experiment - not only because it is open to comment and scrutiny online, but because it is a very difficult experiment to do with many variables to be quantified.

This all takes time and care but above all it takes experience gained working with the experimental set up. I suspect people may be a bit disappointed that there was no positive result. But on a first run? – never happens…

But so far it has been a really great start.

0 #11 Al Potenza 2012-11-13 20:10

"This is why in these types of experiments a Delta T across a boundary layer of the apparatus sometimes is the most accurate"
-
Couldn't agree more. And if the "boundary layer" is dimensionally stable and its thermal conductivity doesn't change, the delta T is a direct measure of heat flux through the layer. You can calibrate as such with your heater wire.

0 #10 J. Andrew Key 2012-11-13 19:31

Can you provide a link to the source code for the data processing tool? I'm interested in porting your algorithm to a FOSS language like Java or Python.

0 #9 Ryan Hunt 2012-11-13 19:22

I have been looking at that inverse correlation too. As t_ambient rises, the T_rise gets smaller, giving the inverse relationship. It takes a little while for the cell to come to equilibrium and give a valid measurement of P_xs after that. I am hoping to see everything relatively steady for 45 minutes before I have a lot of confidence in the output.

0 #8 Andre Blum 2012-11-13 19:15

I buy that. Thanks David.

0 #7 Christian Bistriceanu 2012-11-13 19:15

Guys, looking at the data it look like the P_xs is going up with a lower pressure. :) guess you probably have noticed that too. May I suggest to move the column of the pressure data next to the P_xs one to have a better view on it? Also, has the cell been loaded with with pure H or there are more gases used?

Keep up the good work and I really wish this experiment will be a total success.

Regards,
Christian

0 #6 David Jones 2012-11-13 19:10

Andre

Perhaps this is due to the perspex safety shield not only shielding the researcher for safety but also thermally shielding the Celani tube from the ambient room temperature.

It can be problematic if ambient room temperature is a significant factor in an experiment - because where do you measure it and how uniform is it?

This is why in these types of experiments a Delta T across a boundary layer of the apparatus sometimes is the most accurate.

0 #5 Andre Blum 2012-11-13 18:18

I am a bit worried by the apparent (inverse) relation between P_xs and ambient temperature.
This morning there was a sudden increase in ambient temp, probably due to room heating starting, P_xs dropped significantly. Plotting both signals in a graph shows a very symmetric relation. This is not what I would expect. Could there be an error in the calculation?
Keep up the spirit!

0 #4 Christian 2012-11-13 17:27

Hi guys,

I am trying to download the latest CSV data but the link to the tests is blank? Do I still have the correct link or this is now blank? Here is where I go : hugnetlab.com/hugnetlab/

0 #3 David Roberson 2012-11-13 15:58

You are offering an excellent solution to the large quantity of data overload. As you know, a lot of important information is hidden within the raw data that might prove critical to understanding.

0 #2 Nicolas Chauvin 2012-11-13 15:39

Good point!
We will add a resource page.

Nicolas

0 #1 Ecco 2012-11-13 14:51

Will try this later.

I propose again the creation of a "resource" page where software tools like this one (or the Java one posted in another thread, to make quickly charts of csv files) can be listed together, for people interested in data analysis.

Refresh comments list
RSS feed for comments to this post

Here is your generous contributions so far towards our $500,000 target, thanks everyone! : $45,020 Please Donate
See the current state of our booked costs here