The more I used ESPhome, the more confused I got. So I decided to start over again and provide some examples for things I had understood or which at least I thought I had understood.

I took a Wemos D1 mini board and generated a new yaml to which I just added a wifi sensor, as it's always good to know the wifi strength when testing.

esphome:
  name: eh-test

esp8266:
  board: d1_mini

# Enable logging
logger:

# Enable Home Assistant API
api:
  password: ""

ota:
  - platform: esphome
    password: ""

wifi:
  ssid: !secret wifi_ssid
  password: !secret wifi_password

  # Enable fallback hotspot (captive portal) in case wifi connection fails
  ap:
    ssid: "Eh-Test Fallback Hotspot"
    password: "password"

captive_portal:

sensor:
  - platform: wifi_signal
    name: "WiFi Signal Sensor"
    update_interval: 10s

 As for the wifi credentials, it's necessary to create a secrets.yaml in the same directory as the above yaml file and provide the credentials there:

wifi_ssid: "your_ssid"
wifi_password: "your_secret_password"

In home assistant (HA) this can then be seen as 

(Settings -> Devices & Services -> ESPHome -> Devices)