Julia demo programs

Interested in Julia? Have a look to this presentation.

Warning:
This page was written when Julia Version 0.6 was available. Now Version 1.0 is available, which means that commands and source code need to be update to work if using it. It will be done as soon as possible.

At the moment we just describe the tool that allows to import into a MySQL table a FITS table HDU. You need to have Julia installed. If you do not have it, get it from here.

Mac users:
to be able to run julia from the command line, create this (or a similar) logical link:

sudo ln -s /Applications/Julia-0.6.app/Contents/Resources/julia/bin/julia /usr/local/bin/julia

This assumes you have downloaded “Julia-0.6” and copied in your Application folder.
Remember to check you version with the command ls -d /Applications/Julia*.

All the packages and other information needed by julia will be placed in the directory ~/.julia. Run ls ~/.julia to see what’s in there.

For convenience, the tar file soft_dirs.tar.gz contains the whole Soft tree structure.
If not done yet, download it in the ~/mpe2018 directory (or another you prefer) and untar it:

shell> cd ~/mpe2018
shell> wget https://ross2.oas.inaf.it/imprs-db/soft_dirs.tar.gz
shell> tar zxvf soft_dirs.tar.gz

Importing a FITS table

If you have downloaded the Soft sub-directory Julia, you’ll see in there the program fits2db.jl, otherwise download it from here.
First of all you need to install some packages in Julia before you can run the script:

shell> julia
               _
   _       _ _(_)_     |  A fresh approach to technical computing
  (_)     | (_) (_)    |  Documentation: https://docs.julialang.org
   _ _   _| |_  __ _   |  Type "?help" for help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 0.6.2 (2017-12-13 18:08 UTC)
 _/ |\__'_|_|_|\__'_|  |  
|__/                   |  x86_64-apple-darwin16.7.0

julia>

Now give these commands (the julia> prompt is omitted for an easier cut-and-paste):

Pkg.add("ArgParse")
Pkg.add("NamedTuples")
Pkg.add("MySQL")
Pkg.clone("https://github.com/JuliaAstro/FITSIO.jl.git")
Pkg.build("FITSIO")

Note that this information is also reported in the header of fits2db.jl!
If all the commands complete successfully (otherwise let me know), then you can use it. In a terminal, possibly from the directory where it resides, first of all let’s run it asking for help (note that Julia programs at first run are slow because they need to be compiled!):

shell> julia fits2db.jl -h
usage: fits2db_ln1.jl -u USER -p PASSWORD -d DATABASE [-s SERVER]
                      [-H HDU] [-v] [-h] FITS

positional arguments:
  FITS                  FITS file

optional arguments:
  -u, --user USER       User name
  -p, --password PASSWORD
                        Password
  -d, --database DATABASE
                        Database name
  -s, --server SERVER   Server host (default: "localhost")
  -H, --HDU HDU         HDU to import, starting from 1 (default: last
                        one) (type: Int64, default: -1)
  -v, --verbose         Be verbose
  -h, --help            show this help message and exit

Then launch the program passing the required information. Here we use as input FITS file the XMM DR8 catalogue (in the compact version):

julia fits2db.jl -u mpeusr -p mpe2018pass -d mpe2018db ../../Data/3XMM_DR8cat_slim_v1.0.fits.gz

Of course use the correct path both for the Julia script and the input FITS table!
You will see some output check information about the table, and at last:

531454 record(s) written

Now you are ready to enter the MySQL client terminal and perform queries on the created table:

shell> mysql -u mpeusr -p mpe2018db

show tables;
-- 3XMM_DR8cat_slim_v1_2
-- ...
-- View the number or entries
select count(*) from 3XMM_DR8cat_slim_v1_2;
+----------+
| count(*) |
+----------+
|   531454 |
+----------+

-- Show the table structure
describe 3XMM_DR8cat_slim_v1_2;
-- ...

Annotate the RA and Dec field names: SC_RA ad SC_DEC
Now we can prepare it for SID usage. We choose here an HTM depth 6 indexing. As an exercise can try with another one of your choice, for example HEALPix order 8.

ALTER TABLE 3XMM_DR8cat_slim_v1_2 ADD COLUMN htm6 SMALLINT UNSIGNED NOT NULL;
UPDATE 3XMM_DR8cat_slim_v1_2 SET htm6 = HTMLookup(6, SC_RA, SC_DEC);
ALTER TABLE 3XMM_DR8cat_slim_v1_2 ADD KEY (htm6);

Now the catalogue is ready for all the sky region queries (and more) you want! A few examples:

CALL SID.SelectCircleHTM('', '*', 'mpe2018db.3XMM_DR8cat_slim_v1_2', 'htm6',  6, 'SC_RA', 'SC_DEC', 30, 38, 20, '');

-- another position, order by flux and just the first 5
CALL SID.SelectCircleHTM('', '*', 'mpe2018db.3XMM_DR8cat_slim_v1_2', 'htm6'   ,  6, 'SC_RA', 'SC_DEC', 101, -44.5, 30, 'ORDER BY SC_EP_2_FLUX DESC LIMIT 5');

-- request only some fields, sort by flux
CALL SID.SelectCircleHTM('', 'IAUNAME, SC_RA, SC_DEC, SC_EP_2_FLUX, N_DETECTIONS', 'mpe2018db.3XMM_DR8cat_slim_v1_2', 'htm6'   ,  6, 'SC_RA', 'SC_DEC', 101, -44.5, 30, 'ORDER BY SC_EP_2_FLUX DESC');