PROBLEM: VAGUE COMMAND LINE ARGUMENTS.

Start-EditorServices.ps1 has vague command line arguments

RESEARCH: Start-EditorServices.ps1

[CmdletBinding(DefaultParameterSetName="NamedPipe")]
param(
    [Parameter(Mandatory=$true)]
    [ValidateNotNullOrEmpty()]
    [string]
    $HostName,

    [Parameter(Mandatory=$true)]
    [ValidateNotNullOrEmpty()]
    [string]
    $HostProfileId,

    [Parameter(Mandatory=$true)]
    [ValidateNotNullOrEmpty()]
    [string]
    $HostVersion,

    [ValidateNotNullOrEmpty()]
    [string]
    $BundledModulesPath,

    [ValidateNotNullOrEmpty()]
    $LogPath,

    [ValidateSet("Diagnostic", "Verbose", "Normal", "Warning", "Error")]
    $LogLevel,

    [Parameter(Mandatory=$true)]
    [ValidateNotNullOrEmpty()]
    [string]
    $SessionDetailsPath,

    [switch]
    $EnableConsoleRepl,

    [switch]
    $UseLegacyReadLine,

    [switch]
    $DebugServiceOnly,

    [switch]
    $LanguageServiceOnly,

    [string[]]
    $AdditionalModules,

    [string[]]
    $FeatureFlags,

    [switch]
    $WaitForDebugger,

    [switch]
    $ConfirmInstall,

    [Parameter(ParameterSetName="Stdio", Mandatory=$true)]
    [switch]
    $Stdio,

    [Parameter(ParameterSetName="NamedPipe")]
    [string]
    $LanguageServicePipeName = $null,

    [Parameter(ParameterSetName="NamedPipe")]
    [string]
    $DebugServicePipeName = $null,

    [Parameter(ParameterSetName="NamedPipeSimplex")]
    [switch]
    $SplitInOutPipes,

    [Parameter(ParameterSetName="NamedPipeSimplex")]
    [string]
    $LanguageServiceInPipeName,

    [Parameter(ParameterSetName="NamedPipeSimplex")]
    [string]
    $LanguageServiceOutPipeName,

    [Parameter(ParameterSetName="NamedPipeSimplex")]
    [string]
    $DebugServiceInPipeName = $null,

    [Parameter(ParameterSetName="NamedPipeSimplex")]
    [string]
    $DebugServiceOutPipeName = $null
)

Import-Module -Name "$PSScriptRoot/PowerShellEditorServices.psd1"
Start-EditorServices @PSBoundParameters

This script is just a wrapper for the Start-EditorServices function in PowerShellEditorServices.psd1

RESEARCH: PowerShellEditorServices.psd1

This psd1 (hashtable as a manifest) with the following relevant keys:

# Script module or binary module file associated with this manifest.
RootModule = if ($PSEdition -eq 'Core')
    {
        'bin/Core/Microsoft.PowerShell.EditorServices.Hosting.dll'
    }
    else
    {
        'bin/Desktop/Microsoft.PowerShell.EditorServices.Hosting.dll'
    }

...

# Cmdlets to export from this module, for best performance, do not use wildcards and do not delete the entry, use an empty array if there are no cmdlets to export.
CmdletsToExport = @('Start-EditorServices')

I'm running core on my instance, so we will checkout cre

RESEARCH: core/Microsoft.PowerShell.EditorServices.Hosting.dll

I need to figure out how this module was built.

RESEARCH: PSES/PowerShellEditorServices.build.ps1

Code that publishes the output of the build:

    # PSES/bin/Core
    foreach ($hostComponent in Get-ChildItem $script:HostCoreOutput) {
        if (-not $includedDlls.Contains($hostComponent.Name)) {
            Copy-Item -Path $hostComponent.FullName -Destination $psesCoreHostPath -Force
        }
    }

 The directory with build output is stored at $script:HostCoreOutput

RESEARCH: $script:HostCoreOutput

Definition:

$script:HostCoreOutput = "$PSScriptRoot/src/PowerShellEditorServices.Hosting/bin/$Configuration/$($script:NetRuntime.PS7)/publish"

It is only used once. Maybe we can find more from $Configuration and NetRuntime.PS7?

This doesn't seem to turn up much

Maybe we can get some data from buildinfo?

RESEARCH:CreateBuildInfo

There are 2 variations of CreateBuildInfo:

Option A

Task Build FindDotNet, CreateBuildInfo, {
    # NOTE: We use /p:UseSharedCompilation=false to work around a bug with CodeQL.
    Exec { & dotnet restore $VerbosityArgs }
    Exec { & dotnet publish /p:UseSharedCompilation=false $VerbosityArgs -c $Configuration .\src\PowerShellEditorServices\PowerShellEditorServices.csproj -f $script:NetRuntime.Standard }
    Exec { & dotnet publish /p:UseSharedCompilation=false $VerbosityArgs -c $Configuration .\src\PowerShellEditorServices.Hosting\PowerShellEditorServices.Hosting.csproj -f $script:NetRuntime.PS7 }
    if (-not $script:IsNix) {
        Exec { & dotnet publish /p:UseSharedCompilation=false $VerbosityArgs -c $Configuration .\src\PowerShellEditorServices.Hosting\PowerShellEditorServices.Hosting.csproj -f $script:NetRuntime.Desktop }
    }

    # Build PowerShellEditorServices.VSCode module
    Exec { & dotnet publish /p:UseSharedCompilation=false $VerbosityArgs -c $Configuration .\src\PowerShellEditorServices.VSCode\PowerShellEditorServices.VSCode.csproj -f $script:NetRuntime.Standard }
}

This reveals that there is build information in these files:

• \src\PowerShellEditorServices\PowerShellEditorServices.csproj
• \src\PowerShellEditorServices.Hosting\PowerShellEditorServices.Hosting.csproj

It calls the last one twice, once for core, and once for windows

Option B

Task CreateBuildInfo {
    $buildVersion = "<development-build>"
    $buildOrigin = "Development"
    $buildCommit = git rev-parse HEAD

    # Set build info fields on build platforms
    if ($env:TF_BUILD) {
        if ($env:BUILD_BUILDNUMBER -like "PR-*") {
            $buildOrigin = "PR"
        } elseif ($env:BUILD_DEFINITIONNAME -like "*-CI") {
            $buildOrigin = "CI"
        } else {
            $buildOrigin = "Release"
        }

        $propsXml = [xml](Get-Content -Raw -LiteralPath "$PSScriptRoot/PowerShellEditorServices.Common.props")
        $propsBody = $propsXml.Project.PropertyGroup
        $buildVersion = $propsBody.VersionPrefix

        if ($propsBody.VersionSuffix) {
            $buildVersion += '-' + $propsBody.VersionSuffix
        }
    }

    # Allow override of build info fields (except date)
    if ($env:PSES_BUILD_VERSION) {
        $buildVersion = $env:PSES_BUILD_VERSION
    }

    if ($env:PSES_BUILD_ORIGIN) {
        $buildOrigin = $env:PSES_BUILD_ORIGIN
    }

    [string]$buildTime = [datetime]::Today.ToString("s", [System.Globalization.CultureInfo]::InvariantCulture)

    $buildInfoContents = @"
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT License.
using System.Globalization;
namespace Microsoft.PowerShell.EditorServices.Hosting
{
    public static class BuildInfo
    {
        public static readonly string BuildVersion = "$buildVersion";
        public static readonly string BuildOrigin = "$buildOrigin";
        public static readonly string BuildCommit = "$buildCommit";
        public static readonly System.DateTime? BuildTime = System.DateTime.Parse("$buildTime", CultureInfo.InvariantCulture.DateTimeFormat);
    }
}
"@

    if (Compare-Object $buildInfoContents.Split([Environment]::NewLine) (Get-Content $script:BuildInfoPath)) {
        Write-Host "Updating build info."
        Set-Content -LiteralPath $script:BuildInfoPath -Value $buildInfoContents -Force
    }
}

This reveals that there is data within .\PowerShellEditorServices.Common.props

This also suggests that whatever is in $script:BuildInfoPath contains more build info

RESEARCH: $script:BuildInfoPath

It is resolved on this line:

$script:BuildInfoPath = [System.IO.Path]::Combine($PSScriptRoot, "src", "PowerShellEditorServices.Hosting", "BuildInfo.cs")

Which should resolve to .\src\PowerShellEditorServices.Hosting\BuildInfo.cs

RESEARCH: BuildInfo.cs

This appears to just be a cs file that packs the build number into the published app

RESEARCH: PowerShellEditorServices.Common.props

This appears to just be some kind of .NET manifest

RESEARCH: PowerShellEditorServices.csproj + PowerShellEditorServices.Hosting.csproj

Neither of these 2 provide anything super revealing. So maybe the builder just compiles everything in their parent directories?

Today, I pushed a pull request to their repository adding a script that is a bit more readable and scripter-friendly than the original one they used. I'd love to see it get merged!

When running the initializer script, it runs it in a secondary pwsh instance to prevent the main script from hanging. However, you can't fully kill it from the parent process, because the second script spawns a third mystery powershell instance that doesn't die when you kill the second one... Gotta find a work around to prevent leaks in my scripts :(

Finding the Cause:

• Start Point: PowerShell-Quick-Tray/initialize.pses.server.ps1
  • $PSES_BUNDLE_PATH/PowerShellEditorServices/Start-EditorServices.ps1
  • Gist of the referenced file: [gist]
    • Import-Module -Name "$PSScriptRoot/PowerShellEditorServices.psd1"
    • Gist of the referenced file: [gist]
      • [line from gist]:

      • RootModule = if ($PSEdition -eq 'Core')
            {
                'bin/Core/Microsoft.PowerShell.EditorServices.Hosting.dll'
            }
            else
            {
                'bin/Desktop/Microsoft.PowerShell.EditorServices.Hosting.dll'
            }

        These are good DLLs to possible look at

    • Start-EditorServices @PSBoundParameters
      • Github Search for "Start-EditorServices"
        • C# definition for the command

Couldn't find it at the moment. Trying to increase logging verbosity.

Here are the valid values for loglevel:

https://github.com/PowerShell/PowerShellEditorServices/blob/8e92d8ee7c5d382b1fa0ea143a5dd1d3fe838d80/module/PowerShellEditorServices/Start-EditorServices.ps1#L52

It is also possible that the only way to shutdown the app is via LSP message

Running this code from stackoverflow to get child PIDs of the second powershell instance reveals the PID of the third one, but also reveals that it also spawned a conhost instance.

Gonna try using this process tree killer to kill the server:

https://stackoverflow.com/questions/55896492/terminate-process-tree-in-powershell-given-a-process-id

Ha, ha, ha... lol I'm an idiot

My script starting the Editor Services is another call to pwsh. I'm an idiot.

Also, fun fact. This was staring me in the face the whole time:

Alright, now to test support for it:

• Can I get an executable? yes:
  • Download PSES (PowerShellEditorServices.zip):
    • https://github.com/PowerShell/PowerShellEditorServices/releases
  • Run the following script to start the server:
    • https://github.com/smartguy1196/PowerShell-Quick-Tray/blob/ccbca5ae07259e1a0b9ac370c0bbdf063d8a34d7/initialize.pses.server.ps1
• Can I access stdio? -not powershell readline...
• Can I access the named pipe?
  • Is there a consistent name for the pipe? no it is randomly generated
    • If not, can I write a script that detects it? yes, however it is uses c++ naming convention.
      • Can I write a PowerShell function that gets one under the c# convention? yes, just need to run "($name -split "\\") | select -last 1"
  • Can I write a powershell script that connects then disconnects from it?
  • Can I send anything to the server via powershell?
    • If yes, can I send the contents of a JSON file via an LSP request to the server?
• Can I get a unit testing script running? yes, I'm writing one
  • Posted yet?

If this works, I need to make a comprehensive list of every Request and Notification that I need to be able to send to the server.

Parsing responses will come with time, as I can just dump unknown responses to a text file and read them later.

https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#initialize

Can use this to start testing out approaches in using this code. The next log will be dedicated to testing out different approaches.

Features of LSP Video:

Keynotes:

Using LSP in my utility, will allow me to easily port my utility to other shell languages like bash.

• While loop 2-way pipe WRITER+client:
  • https://gist.github.com/scudette/b6a50f9756f58ed93c9f0f3bdcc01072
• Function-based 1-way pipe WRITER+client:
  • https://www.powershellgallery.com/packages/PoshInternals/1.0/Content/NamedPipes.ps1
• While loop 2x 1-way pipe (READER+WRITER)+server:
  
  • https://majornetwork.net/2021/05/inter-process-communication-with-named-pipes-between-python-and-powershell/
• Another server with 1-way WRITER+client:
  • https://gbegerow.wordpress.com/tag/powershell-named-pipes/
• C# server with C++ client:
  • https://dev.to/gabbersepp/ipc-between-c-and-c-by-using-named-pipes-4em9
• C++ server with C# client:
  • https://stackoverflow.com/questions/9676343/how-to-work-with-named-pipes-c-server-c-sharp-client

DOCUMENTATION: https://microsoft.github.io/language-server-protocol/

• LSP uses JSDoc for documentation
• Change Log:
  • https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#version_3_17_0
• `\r\n` is used as the main delimiter in the protocol
  • Used to terminate field entries in the header
  • Used again to separate the header from the content
• Header has only 2 supported fields:
  • Content-Length
  • Content-Type
    • See the following for valid content-types:
      • https://stackoverflow.com/a/23714451
      • https://greenbytes.de/tech/webdav/draft-ietf-httpbis-p2-semantics-26.html#rfc.section.3.1.1.5
• Content: goes over this in detail, but is pretty much just the JSON response/request
• After a quick review of JSON-RPC used in the spec, JSON-RPC isn't JSON, but JSON-based?? Great, now I gotta learn another language :(
  • JSON-RPC spec:
    • https://www.jsonrpc.org/specification
  • Nevermind, the API for it is just written in TypeScript for some ungodly reason (Do I really need type safety for an API description??)
• The spec enforces the use ECMAScript Regex in messages. (Will this be an issue in PowerShell or C#?)
• Holy moly this doc is humungous

Notes:

LSP:

• LSP is new (orignated in early 2016 by Microsoft)
• List of LSP Servers and assessment of how well they are maintained:
  • https://langserver.org/
• Spec: https://microsoft.github.io/language-server-protocol/
• LSP is a JSON-RPC protocol

PowerShell Editor Services:

• The only listed LSP server for PowerShell is the one maintained by Microsoft
• Powers the PowerShell Extension for VSCode
• You can connect to it using Named Pipes or Stdio
• Documentation:
  • Index:
    • https://github.com/PowerShell/PowerShellEditorServices/blob/main/docs/index.md
  • User Guide:
    • https://github.com/PowerShell/PowerShellEditorServices/blob/main/docs/guide/introduction.md
  • API Reference (.NET API??):
    • https://github.com/PowerShell/PowerShellEditorServices/blob/main/docs/api/index.md
    • NuGet package:
      • https://www.nuget.org/packages/Microsoft.PowerShell.EditorServices/
    • I believe this just controls the services aspect. Maybe I can start it and manage it in .NET?
    • FuGet.org page:
      
      • https://www.fuget.org/packages/Microsoft.PowerShell.EditorServices/0.6.2.389
  • "The Host Process" - what is this? Got it! This is the doc on how to use STDIO
    • https://github.com/PowerShell/PowerShellEditorServices/blob/main/docs/guide/using_the_host_process.md
    • states that the only way to interact is with stdio despite the README which says that named pipes are supported??
    • goes over LSP in detail
    • PowerShell Extension Terminal uses this method
  • Named Pipes API (which there is no document for this, guess I gotta learn how to use named pipes lmao)
    • Visual Studio Code, Vim, and IntelliJ use this method - Vim uses this?? Isn't Vim old????