Today I noticed the FAN speed was switched back to 1 when it already was at 1. Therefore I added this to the automation script:

  condition:
  - condition: numeric_state
    entity_id: sensor.fan_value
    above: '1'

Now it won't run the automation if the FAN speed is already at level 1.