Demo Discussion
Forum Config Examples Contributions Vulnerabilities
  Contributions to ELOG, Page 3 of 6  Not logged in ELOG logo
ID Date Author Author Email Categorydown Subject Status Last Revision
  37   Tue Aug 23 19:45:16 2011 Stuart Wilkinsswilkins@bnl.govScriptELOG Mail ParserAlpha 

 Hi!

I have attached a small script which can process an e-mail and submit this e-mail as a log entry. This is alpha at the moment but if you have suggestions or find bugs please let me know. 

The easiest way to process this is to use the ".forward" file in unix. If this file contains the line:

| "/usr/bin/python /home/tardis/elogmail/process.py --log /home/tardis/elogmail/mail.log -r -u http://localhost:8080/TARDIS -a Category=email -a Type=Log"

This will pipe the incoming mail to the e-log running on the localhost with the experiment TARDIS. Attributes can be set using the "-a" switch like in the elog command line utility. To get help with all the options please run the script with the command:

python process.py -h

which will give help.

Any questions, please don't hesitate to contact me, and I will do my best to help.

Thanks again to Stefan for a great application.

Best wishes,

Stuart

 

Attachment 1: process.py.gz
Attachment 2: forward.gz
  38   Fri Sep 9 12:08:26 2011 Andreas Luedekeandreas.luedeke@psi.chScriptcreate incremental daily backups of logbooks for Unix/LinuxBeta 
A very simple backup utility for elog logbooks.
If you want to recover your logbook from corrupted entries, an incremental backup comes in handy:
you can just replace the corrupted files with the previous versions.
Below is a very simple wrapper to create backups of the logbook into some local directory (/logbooks_backup in
the example script). The second wrapper lists the files in a given ".tar" archive.

You would run the backup from a cron job with the line:
1 0 * * * /usr/local/elog/elog_backup_daily

This will create daily backups of modified logbook and config files and a weekly backup of the password file
/usr/local/elog/passwd.txt (if you wonder why: our passwd.txt file has more than 1 MByte and of course it
changes with every login).

elog_backup_daily (copy to /usr/local/elog) script to create tar file and remove tar files older than 90 days
exclude-logbooks  (copy to /usr/local/elog) list of files to be excluded in backup (thumbnails, etc.) 
elog_backup_listfiles (copy to /usr/local/bin) shows content of a given tar file (omits directories in list)
Attachment 1: elog_backup_daily
#!/bin/sh
# source file directory
srcd=/usr/local/elog/

# backup all files newer than 25 hours
date=$(date -d "-25hours" "+%Y%m%d %H:%M")
# backup file directory
tard=/logbooks_backup
# backup tar file name
tarf=$tard/$(date +%Y%m%d_%a.tar)
# do not backup files that match patterns in this file
excf=/usr/local/elog/exclude-logbooks
# create backup
cd $srcd
tar --ignore-case -X $excf --newer "$date" -cf $tarf . logbooks/*

# copy passwd.txt
if [ "$(date +%u)" -eq 1 ]
then
	passwd=$tard/passwd_$(date +%Y%m%d_%a.txt)
	cp $srcd/passwd.txt $passwd
	gzip -9 $passwd
fi
# delete all backups older than 90 days
find $tard -mtime +90 -name "*_???.tar"           -exec rm -f {} +
find $tard -mtime +90 -name "passwd_*_???.txt.gz" -exec rm -f {} +
Attachment 2: exclude-logbooks
*.png.png
*.gif.png
*.jpg.png
*.pdf.png
oldLogbooks/*
logbooks/elog.log
passwd.txt
ssl/*
logbooks/Backup/*
logbooks/old/*

Attachment 3: elog_backup_listfiles
tar -tf $1|grep -v "^.*/$"
  40   Mon Jan 7 08:45:10 2013 Bruce Weberbruce.weber@inmarsat.comScriptServer time offsetStable 

Our server runs on local time, however, we require log entries to be in UTC (-0800hrs) - is there a command to achieve this?

Your assistance will be much appreciated

Thanks

  41   Mon Jan 7 16:37:21 2013 Stefan Rittstefan.ritt@psi.chScriptRe: Server time offsetStable 

Bruce Weber wrote:

Our server runs on local time, however, we require log entries to be in UTC (-0800hrs) - is there a command to achieve this?

Your assistance will be much appreciated

Thanks

Have you tried

 

Subst <attribute> = $utcdate

 

Probably you need a dedicated attribute of type "datetime" and have to play with the "Date format" options.

  44   Tue Jul 8 15:43:21 2014 TimStimpie_s@yahoo.comScriptRe: Custom input forms implementationStableTue Mar 19 13:18:33 2013 by Stefan Ritt


Stefan Ritt wrote:

Dear ELOG users,

starting with SVN revision 2328, custom input forms are implemented. This allows application specific formats for check lists etc. In our specific case we had to implement a shift check list, which was quite long. Furthermore the check list should be optimized for an iPad, which we take in the field and record various checks and readings (in our case some gas pressure gauges at the PSI particle accelerator). Since the standard ELOG interface was too inflexible, a completely hand-written form was needed. The form can be activated by the new configuration options Custom New Form, Custom Edit Form and Custom Display Form, one for a new entry, an entry to edit and and entry to display. In our case we used the same form for all three cases. This is how the shift check list looks under the Safari Browser on a PC:

Capture.png 

And here is how it looks on the iPad:

IMAG0036.jpg

Each section can be collapsed and expanded (blue arrows at the left), and various internal checks are made before the check list can be submitted.

Implementing such forms is however more something for the advanced user, since you have to hand-write HTML with CSS and JavaScript code. It can then however be a powerful method for check lists. Please find in the attachments the elogd.cfg configuration for that logbook and the shiftcheck.html source code file. It is a bit complicated since the page is a static page, elogd just serves it from the file. This requires all the dynamic functions to be implemented inside the HTML file with JavaScript. To display an entry for example, the JavaScript loads the raw data with the "?cmd=Download" command and the populates the form fields. The collapsing and expanding is done by using CSS properties. The integrated style sheet was optimized for the rendering on an iPad. Rather large fonts were chosen so that the items can be checked easily with your finger tips. Various parameters are sent between the browser and the elogd program via hidden fields and cookies. So only something for experts! But if you go through the effort and hand-write the form, it can be very handy. Note that you have to upgrade to SVN revision 2328 for the three new options.

 

 

That's awesome!! Didn't know ELOG was still being developed!!  I was looking for this ...been trying this version now with
supplied example but there's files missing which are being referenced from the html file.

Anyway it would be nice to know a little more on how to use this and how it incorporates into ELOG.

Is some more work being done on this version ?

Thankx heaps for this great enhancement although now it's still needs a lot of figuring out on how to do this.
 

  45   Thu May 7 09:14:25 2015 Christof Hankehanke@rzg.mpg.deScriptsystemd-unit fileStable1

sample systemd-unit file.

Use it as you like.

Attachment 1: elogd.service
[Unit]
Description=ELOG Daemon
Wants=network-online.target
After=network-online.target

[Service]
Type=forking
ExecStart=@PREFIX@/elogd -D -c @PREFIX@/elog/elogd.cfg
StandardOutput=null

[Install]
WantedBy=multi-user.target
  49   Wed May 11 09:35:23 2016 Hanno Perreyhanno.perrey@nuclear.lu.seScriptRe: Re: Custom input forms implementationStableWed May 11 09:43:34 2016 by Hanno Perrey
Stefan Ritt wrote:

Dear ELOG users,

starting with SVN revision 2328, custom input forms are implemented. This allows application specific formats for check lists etc. In our specific case we had to implement a shift check list, which was quite long. Furthermore the check list should be optimized for an iPad, which we take in the field and record various checks and readings (in our case some gas pressure gauges at the PSI particle accelerator). Since the standard ELOG interface was too inflexible, a completely hand-written form was needed. The form can be activated by the new configuration options Custom New Form, Custom Edit Form and Custom Display Form, one for a new entry, an entry to edit and and entry to display. In our case we used the same form for all three cases. This is how the shift check list looks under the Safari Browser on a PC:

Capture.png

And here is how it looks on the iPad:

IMAG0036.jpg

Each section can be collapsed and expanded (blue arrows at the left), and various internal checks are made before the check list can be submitted.

Implementing such forms is however more something for the advanced user, since you have to hand-write HTML with CSS and JavaScript code. It can then however be a powerful method for check lists. Please find in the attachments the elogd.cfg configuration for that logbook and the shiftcheck.html source code file. It is a bit complicated since the page is a static page, elogd just serves it from the file. This requires all the dynamic functions to be implemented inside the HTML file with JavaScript. To display an entry for example, the JavaScript loads the raw data with the "?cmd=Download" command and the populates the form fields. The collapsing and expanding is done by using CSS properties. The integrated style sheet was optimized for the rendering on an iPad. Rather large fonts were chosen so that the items can be checked easily with your finger tips. Various parameters are sent between the browser and the elogd program via hidden fields and cookies. So only something for experts! But if you go through the effort and hand-write the form, it can be very handy. Note that you have to upgrade to SVN revision 2328 for the three new options.

 

Thank you very much for this nice example!

I found a little issue with newer ELOG versions: if the authentication is done via session cookies, the submission of the checklist will fail with the error "Cannot open file passwd". This is actually misleading as the cause of the error is the empty user name submitted (since the cookie storing user name and pwd hash is never created) and not the password file itself. This can be worked around by removing the user name and pwd fields in the html form before submitting in case there is a SID cookie around.

The attached shiftcheck.html contains this workaround and seems to be functioning fine on my installation (ELOG 3.1.0).

 

Cheers,

Hanno

Attachment 1: shiftcheck.html
  50   Tue Jul 26 10:16:22 2016 Andreas Luedekesimon.ebner@psi.chScriptPython module to read/write/edit/reply/delete ELOG entriesStableTue Jul 26 10:26:34 2016 by Andreas Luedeke
Hello Everybody!
I would just like to inform you that there is a new Python module available to access and work with ELOG (read/write/edit/reply/delete).
Currently the module is only supported for Python version 3. The package is pure Python and has no special dependencies. Therefore you can use the package on any machine.

The documentation you can find on https://github.com/paulscherrerinstitute/py_elog

If you want to install the package in an Anaconda on your local machine just use
conda install -c paulscherrerinstitute elog
(also works from outside PSI - the package is available at https://anaconda.org/paulscherrerinstitute/elog)

If there are issues with the package please use the issue tracker that comes with the github repository
https://github.com/paulscherrerinstitute/py_elog/issues or let me know.
Best
Simon
  52   Fri Sep 15 17:19:47 2017 Andreas Luedekeandreas.luedeke@psi.chScriptCheck logbook files for wrong referencesStableFri Sep 15 17:41:41 2017 by Andreas Luedeke
You can run this little script to check if all entries referenced "In reply to:" do actually exist.
To use it, you first need to "cd" to your logbook directory ("cd /usr/local/elog/logbooks") and then run it without arguments "logcheck".
If it finds references pointing to a missing entry, it'll print the path to the file with the offending reference and some lines. For example:
### error: reference to entry 146, that exists 0 times. Reference is:
### Proscan/2012/120507a.log-<p>[...].</p>
### Proscan/2012/120507a.log-$ @ MID @ $: 147
### Proscan/2012/120507a.log-Date: Mon, 07 May 2012 13:44:03 +0200
### Proscan/2012/120507a.log:In reply to: 146
### Proscan/2012/120507a.log-Wann: 1336373261
### Proscan/2012/120507a.log-Autor: [...]
### Proscan/2012/120507a.log-Eintrag: Problem
[...]-

Very often this happens if an entry is deleted AFTER someone already replied to it. Normally that is no problem, but in some cases you might get infinite loops and that causes ELOG to hang. The script is not checking for loops, but wrong references might give you a hint where to look.

The script will print duplicate entries as well, if the referenced entry exist more than once.

Cheers, Andreas

PS: never include the string "$ @ MID @ $:" without spaces in an ELOG entry: apparently ELOG cuts off all text from that token on.

Attachment 1: logcheck
#!/usr/bin/perl
$cmd=q/find * -noleaf -maxdepth 0 -wholename Backup -prune -o -wholename en -prune -o -type d -exec printf '%s:' {} + /;
#print $cmd;
$logf=q/*\/[0-9][0-9][01][0-9][0-3][0-9]a.log/;
open(INP,$cmd . "2>/dev/null |") || die "can't open " . $cmd . ": $!";
$list=<INP>;
close(INP);
#print "list=" . $list ."\n";
foreach $dir (split(":",$list)) {
    chomp $dir;
    if (length($dir) > 1) {
        print "Check \"" . $dir . "\"\n";
        $files=$dir . "/" . $logf;
        $cmd="grep ^In.reply.to: " . $files . " 2>/dev/null | cut -d \" \" -f 4 | sort -nu |";
        open(INPB,$cmd) || die "can't open " . $cmd . ": $!";
        while (<INPB>) {
            chomp;
            $id=$_;
            $cmd="grep '[\$\]\@MID\@[\$\]:.$id\$' $files |";
            open(INPC,$cmd) || die "can't open " . $cmd . ": $!";
            $c=0;
            while (<INPC>) {$c++};
            close(INPC);
            if ($c != 1) {
                print "### error: reference to entry $id, that exists $c times. Reference is:\n";
                $cmd="grep -C 3 '^In.reply.to:.$id\$' $files |";
                #print $cmd;
                open(INPD,$cmd) || die "can't open " . $cmd . ": $!";
                while (<INPD>) {
                    print "### ".$_;
                }
                close(INPD);
            }
        }
        close(INPB);
    }
}
  53   Sat Sep 16 15:47:16 2017 David PilgramDavid.Pilgram@epost.org.ukScriptRe: Check logbook files for wrong referencesStableSat Sep 16 15:52:58 2017 by David Pilgram

I had to modify the script because I'm still on elog 2.9.2, where there are not subdirectories (by year) for
 each logbook.  Line 5 had to be changed to  

$logf=q/[0-9][0-9][01][0-9][0-3][0-9]a.log/;

to do this.

Once done, I found the deliberate orphan script that I had put in to test, and rather too many other orphans than I had expected.  One or two I cannot explain.   If I had clicked on any of those entries elog would have gone into infinate loop.

A very useful utility.  Thanks Andreas!

Andreas Luedeke wrote:
You can run this little script to check if all entries referenced "In reply to:" do actually exist.
To use it, you first need to "cd" to your logbook directory ("cd /usr/local/elog/logbooks") and then run it without arguments "logcheck".
If it finds references pointing to a missing entry, it'll print the path to the file with the offending reference and some lines. For example:
### error: reference to entry 146, that exists 0 times. Reference is:
### Proscan/2012/120507a.log-<p>[...].</p>
### Proscan/2012/120507a.log-$ @ MID @ $: 147
### Proscan/2012/120507a.log-Date: Mon, 07 May 2012 13:44:03 +0200
### Proscan/2012/120507a.log:In reply to: 146
### Proscan/2012/120507a.log-Wann: 1336373261
### Proscan/2012/120507a.log-Autor: [...]
### Proscan/2012/120507a.log-Eintrag: Problem
[...]-

Very often this happens if an entry is deleted AFTER someone already replied to it. Normally that is no problem, but in some cases you might get infinite loops and that causes ELOG to hang. The script is not checking for loops, but wrong references might give you a hint where to look.

The script will print duplicate entries as well, if the referenced entry exist more than once.

Cheers, Andreas

PS: never include the string "$ @ MID @ $:" without spaces in an ELOG entry: apparently ELOG cuts off all text from that token on.

 

ELOG V3.1.5-2eba886