Dan Maloney
I think it's C++ on CentOS 7
actually, guess what it was
Here are the libraries for data analysis: https://root.cern/
what was the mean time between failures when LHC started in 2008? The first days?
I love C++ and Python.
any guesses?
What about something where even C won't do? VHDL, Verilog, what are you using for FPGAs?
Two weeks?
1 second?
Three months?
now, 15 years later, when everything is tuned and works well, we can get even 2 weeks
Also, how are you coping with the molasses-like nature of light? That thing is s-l-o-w. How do you compensate for that in the signal speed?
but what it was then?
4 hours?
about 5 minutes. Impressive for such a complex system. You can have MTBF of millions of hours. Plug it in million times and here we go
Woah
5 minutes. By the first week of running, it was about 1 hour. People working days and nights. That was the best times I have ever had. Hacking
hacking such a machine. I still get goose bumps thinking about it even now
very high caliber hacking
Understandably.
now the big lady is running reliably and quitely. We work on many smaller machines and projects
oh, you call it big lady?
Isn't there a Really Big Lady in the works now?
sometimes so reliably, that I start to miss the nigh calls. LHC did not call for 3 months. What is going on :-)
How does the LHC behave when there's an earthquake in Europe?
lol.
it's not afraid of earthquakes. but it shivers in fear when a leap second arrives :-)
we are restarting the complex as we speak. The Linacs are already running, it will take about 1 week for each subsequent machine. LHC is foreseen to have first beam the week before the easter
@simon.dancose yes, LHC can see all Earth movements
How do you compensate for them?
The Moon, water level in the Geneva lake, and all the Earth quakes.
Or are they just seen in the data?
The moon and water is slow. Out of 9000 magnets in LHC, maybe 7000 are correctors. The moon phases are even programmed in the control system as a real time feed forward correction
aperture where beam circulates in LHC at high energy is about 1x1mm. Any movement is visible
cool, I figured so. Moon cycle needed pre-planned compensating.
first the beam is scraped at collimators, for for a bigh earthquake, like the one from Turkey which was visible even by my pendulum clock, that would be an instant beam dump
In case you missed it:
https://hackaday.com/2023/01/28/an-atomic-pendulum-clock-accurate-enough-for-cern/
An Atomic Pendulum Clock Accurate Enough For CERN
That big grandfather clock in the library might be an impressive piece of mechanical ingenuity, and an even better example of fine cabinetry, but we'd expect that the accuracy of a pendulum timepiece would be limited to a sizable fraction of a minute per day.
and indeed, there are sensors and accelerometers all over the place. For example the huge earthquake in Indonesia at the beginning of 2000's was already recorded by the Atlas sensors
https://cds.cern.ch/record/824438?ln=en
THE ASIAN EARTHQUAKES DETECTED IN THE ATLAS CAVERN
At the end of December, mysterious vibrations were picked up by the deformation sensors under the feet that are to support the ATLAS detector. It transpired that they had detected waves produced by the earthquakes responsible for the terrible tsunami in Asia.
But other accelerators in the world can see these too. For example in DESY in Hamburg, they can see when people go to work during the day
when you do an emergency beam dump, do the proton packets just get released into a slab of lead or some other absorbent material? how long till that target is able to be handled?
Previous accelerator in the LHC tunnel (the LEP) showed beam instabilities reflecting the schedule of the Geneva-Lyon TGV line :-)
oh no. The LHC beam can make it through 50 meters of concrete... There is a 900 ton graphite/steel block to absorb it
https://lhc-machine-outreach.web.cern.ch/components/beam-dump.htm
LHC beam dumps
The challenge The nominal LHC beam contains an unprecedented stored energy of 350 MJ, contained in 2808 bunches with a beam sigma of the order of 0.3 mm. The extremely high destructive power of such a beam imposes an external dump, where the beam must be extracted completely from the LHC, diluted to reduce the peak energy density and then absorbed in a dedicated system.
What would happen to the concrete? Molten-through capillary-sized hole?
the beam carries so much energy, that it needs to be spread while dumping. Otherwise it will destroy the target. Some people say there will be plenty of very radioactive diamonds in the beam dump after the life time of the machine :-)
It's like a runaway truck ramp for protons
"The extracted beam is swept in a quasi-circular figure by two sets of orthoganally deflecting dilution kickers"
The beam has a kinetic energy of (now) more than 400 MJ. That is a runaway train. Not a truck. And needs to be disposed in 89 microseconds
Imagine what a train would do when crashing into a barrier. But this is 10^14 protons
same macroscopic effect
Do you have a favorite experiment? Or a finding that you're particularly proud of?
is there any video available of a dump event on the absorber cylinder?
or a swarm of 10^14 mosquitos, each flying at a pretty badass mosquito speed :-)
Daniel, thank you so much for your insight!
@Rosy Schechter I have spent a lot of time with LHC. The stabilizing system is my baby and I have feelings about it. But lately, I started to like very much the other end. My collegues from the operations group say it is a machine at the end of the food chain. The smallest one. But there are many scientists coming from many institutes. And they have say 1 week of time to install their experiment and collect all data
the atmosphere is very different from LHC. Those are big fish. A lot of people involved, a lot of competition. The Isolde is exactly the opposite.
right on. looking it up. thank you!
You help the users to achieve their goals, help them to do their experiments. They are happy, thankful, often bring a good bottle to the control room at the end of their run
As will I.
that's got to be really satisfying!
Agreed.
@Rosy Schechter trust me, it is. A very different feeling. Both are good. But very different. While in LHC I am expected to solve any problem in 1 hour regardless, this is an expert service. You do your best, because you want to help them
and it is very nice to collaborate with different people every time. I like both. With LHC we are like a family. The operators rely on experts and the experts rely on operators.
Is our time up? Thank you, Daniel; this was a most intriguing incite into LHC operation.
don't end it. We have just started :-)
time flies fast. Indeed.
Thank you, Daniel!
Holy cow, I just looked at the clock and saw that it's after 1:00. I'm not going to enforce Swiss precision on this end of the chat, but if Daniel has somewhere else to be, we'll give him the chance to log out. Otherwise, we can keep chatting!
Time flies. There is a rotting clock somewhere here.
Thank you, Daniel.
thank you so much @daniel valuch!
@Dan Maloney I just need to move 5 meter to bed. So no rush
Unless the Big Lady calls ;-)
it is a nice concept here. I recall the times of IRC
dromant https://op-webtools.web.cern.ch/vistar/vistars.php
The Big Lady is off for now.
Yeah, that's why we keep doing it this way. A little retro vibe is nice now and then
all is cold and tests are running already. It takes weeks to restart
6/8 sectors are at 1.8K, 1-2 and 3-4 is around 4K. Will need few more days
so you work on ISOLDE now, no longer on call for main beam support, is that correct?
https://op-webtools.web.cern.ch/Vistar/vistars.php?usr=LHC2
Vistars
as an expert for systems, you are always on call. There is also a stand-by service, where people must be reachable and come on site within 45 minutes. But those are expected to solve only a limited subset of problems
Thank you @daniel valuch and everyone who participated!
are you on the french or swiss side?
there is always an army of experts, people who designed and built the particular systems, who can be contacted in case the stand-by is not able to resolve the problem
I have the office on the French side, I also live in France. But we commute to the Swiss side multiple times a day. There are labs, cafeteria, other services
btw, the orange pipes at the ceiling are the RF feed coax lines for the SPS. Carrying 1 MW of RF at 200 MHz, each.
coax or waveguides?
these are coax
yes, that is output of the 6x 1MW amplifiers. Two of those are solid state. A transistor amplifier which needs its own building
coax, as the frequency is low. In LHC we run at 400MHz, but there we use waveguide alredy. 58x24cm. Heavies frequencies in the industry
I need to have a crane operating license. I got it one day before becoming a professor. Just a different kind of exam :-)
Why do you need a crane license?
how does the crane recognize if you have a licence? Our forklift didn't ask.
High power RF is a heavy industry. Look at the photos... 1 meter of waveguide weights 50 kilos
@Thomas Shaddack the crane does not recognize, but the safety people do :-)
plus you don't want to smash your multi million worth equipment
I was at a TED talk at CERN a few years ago, got to look at the surroundings, but notthing below ground so to speak.
getting underground is almost impossible for visitors. During shutdowns, you can visit the experiments underground. It is very impressive. But to get to the machine is impossible
@Thomas Shaddack to answer your HDL question: yes, most hard real-time controls are done on FPGAs (hard RT = determinism from 1ms to anywere near a fraction of a nanosecond). CERN is mostly VHDL, but there are some misfits that use SystemVerilog or even things like Migen/Misoc/LiteX.
I recommend to visit, also the surface installations are nice to see https://visit.cern
DEFINI
the next long shutdown will be in about 3-4 years time, usually an open days is organized. Then people can visit also the underground facilities. Last time about 120000 people came
my favourite part getting people underground was asking the people in the elevator. Who wants to press a button at CERN? One or two rose their hand. So I proposed to press the -1 button in the elevator. They said they will not wash their finger :-)
Discussions
Become a Hackaday.io Member
Create an account to leave a comment. Already have an account? Log In.