Panoramische camera`s en het Astro

ASTRO-WISE Science
“A new approach to astronomical
archiving & researching for the dataflooded era”
Gijs Verdoes Kleijn
OmegaCEN,
Kapteyn Astronomical Institute,
University of Groningen
The OmegaCEN team at Kapteyn
•
•
•
•
•
•
•
•
•
•
•
•
Edwin Valentijn (lead)
Kor Begeman
Danny Boxhoorn
Erik Deul (Leiden)
Ewout Helmich
Philippe Heraudeau
John MacFarland
Michiel Tempelaar
Gijs Verdoes Kleijn
Ronald Vermeij
Willem Jan Vriend (Lofar)
Kovac, Schneider, Sikkema (PhDs)
Archiving your vision
• Camera in front of eye:
–
–
–
–
–
–
–
resolution ~5/60 degree (my eye)
Field of view 902 degree
2x2 pixel sampling per resolution element
#pixels= 21602
Dynamic range: ~103-5: take 216
One image ~9Mbyte
One image/sec for 70years: 18Pbyte
• Storage cost (at 0.33euro/Gbyte)~ 6million euro
– ~1010-11 neurons
Brain does something smart…
……intensive linking is a key ingredient.
If vision archive were implanted for re-analysis
• Applying improved analysis
– (Raw data: improved calibrations as well)
• Analysis of previously deemed uninteresting parts
of data
• Variability analysis
• if archives from different persons are combined
– Denser coverage in space+time
– 3D construction of 2D view
• If fly’s UV eye archives are added:
– Same object at different wavelengths
Aim: exploit data for purposes or in ways not
yet conceived ...
... flexible, internally linked, information
system required
Key properties for intelligent information system
•
Adaption: facilitate changes
1.
2.
3.
•
Learning: “more and better”
–
–
–
–
•
Small Individual research projects
Large projects with many collaborators
Everywhere: federation
–
•
take advantage of existing results (from you or others)
Fast (re-)reduction and (re-)analysis
From quick-look to high-quality results
Not only more but also better data over time: ‘accumulation of knowledge
Anybody
–
–
•
Changes due to improved encoded methods
True physical changes of parameter values (e.g., change in instrument/atmospheric
properties)
Improved insight in 2 or 3
federation of storage and computing capabilities
Scalability
–
No limits due to design for storage, databse power/storage processing power,…
A federated ‘brain’ interacting with many users
The Astro-Wise Environment
a new archive/research tool for
astronomical wide-field imaging
Status
•environment is working
•First paper using Astro-Wise based results is out
•Expansions and improvements on-going
The Astro-Wise Environment
User
Python prompt (awe>)
Web interfaces
“Interpreting”
Database
“Archiving”
Data server: images
calibration & science
From raw to reduced
•metadata of images
•derived data from images
(source lists)
•=‘spider spinning AWE web’
•Contains ALL input/output
“Processing”
CPUs +algorithms
Key properties for intelligent information system
•
Adaption: facilitate changes
1.
2.
3.
•
Learning
–
–
–
–
•
Each piece of information carries tag of ‘ownership’
Small Individual research projects
Large projects with many collaborators
Anywhere: federation:
–
–
•
take advantage of existing results (from you or others)
Outdated not-yet existing data
Fast (re-)reduction and (re-)analysis
Not only more but also better data over time: ‘accumulation of knowledge‘
Anybody:
–
–
–
•
Changes due to improved encoded methods
True physical changes of parameter values (e.g., change in instrument/atmospheric
properties)
Improved insight in 2 or 3
Own developed compute-grid and storage-grid
Nodes:active={Kapteyn, Bonn} almost={Munich, Paris,Naples}, in
progress={Nijmegen, Leiden, Santiago})
Scalability: performance proportional to I/O speed and processing power.
A federated ‘brain’ interacting with many users
Paradigm shift
AWE
“Classical”
Information
system
Dynamic archive
Releases
‘Continous’
releases (VO)
End result ‘pulled’
via linking of all
processing
input/output
Fixed releases
Static archive
Raw data
‘pushed’ through
pipeline to endproduct
Science projects with AWE
• PhDs Sikkema, Kovac,
Schneider: galaxy surveys
with WFI, WFC, MDM
• Test projects
– Variable sources around
CenA
2dF
Cen A
• light curves (Δmag~0.03) for 2x104
objects around Centaurus A
• Valentijn
– Asteroid detection
• Detections in WFI image from
catalog of ~105 numbered asteroids
• Jeffrey Bout (student), myself
Asteroids
Large public data projects with
AWE using OmegaCAM at VST
• KIDS (PI: Kuijken): ESO Public Survey
– Weak lensing; high-z QSOs; galaxy/cluster
evolution; baryon oscillations;
– 5000 deg2 u,g,r,i, ~500 nights
KIDS North
KIDS South
GTO science with AWE
using OmegaCAM at VST
• OmegaWhite (PI: Groot):
– discover Galactic Population of
ultracompact binaries from periodic
(<2hour) light curves
– 150 deg2 @ b=±5o; Sloan g’ 40sec
exposures & additional ugriz’ coverage.
GTO science with AWE
using OmegaCAM at VST
• OmegaTranS (Saglia; Snellen; Alcala)
– Searching for planet transits (15-20 new
transits expected in first year)
– ~1 order more powerful than OGLE-III
GTO science with AWE
using OmegaCAM at VST
• VESUVIO (PIs: Valentijn, Capaccioli)
– Galaxy/cluster evolution
– Horologium Supercluster: 100 deg2
medium deep (r'<25mag); ugriz
– Hercules Supercluster: 12 deg2 deep
ugriz+Hα
Key ingredients to achieve the
Astro-Wise environment
– Strict global data acquisition and processing model
– data model -> object model
– storing all I/O in single (distributed)
database
– Database environment exploits
• OOP inheritance (Python)
• Complete linking (associations, references)
Conclusions
• New analysis environment for wide-field imaging in
operation by OmegaCEN
– Key ingredient: dynamic database
• Containing all I/O
• Fully linked data
• Versatile: could be used for other kinds of data
– Collaboration with LOFAR on-going
• Large science projects with AWE when OmegaCAM
starts operations (early 2007)
• To find out more
– Visit : www.astro-wise.org;
– download/get now 2 page overview article
Contribution to LOFAR project