IntelliSense fails occasionally - again

[PrzemyslawKlys]

Issue Description

I am experiencing a problem with... IntelliSense failing.

1591449835-872343d1-59b3-436d-a112-c71c820037db1591449833798.zip

I feel a bit stupid filling this again but again got this issue. I am not sure if it's:

• Language Features still stops working after a while and never recovers #2627

Or it's more about this:

• Workaround when a module takes too long to run Get-Help #2737

But I still see hiccups that never recover. I deleted a lot of offending PowerShell modules, but since I have no clue about reasons it never recovering it's hard to say.

If you tell me what to look for, check I'm happy to do my own investigations.

Attached Logs

Follow the instructions in the README about
capturing and sending logs.

Environment Information

Visual Studio Code

Name	Version
Operating System	Windows_NT x64 10.0.19041
VSCode	1.45.1
PowerShell Extension Version	2020.6.0

PowerShell Information

Name	Value
PSVersion	5.1.19041.1
PSEdition	Desktop
PSCompatibleVersions	1.0 2.0 3.0 4.0 5.0 5.1.19041.1
BuildVersion	10.0.19041.1
CLRVersion	4.0.30319.42000
WSManStackVersion	3.0
PSRemotingProtocolVersion	2.3
SerializationVersion	1.1.0.1

Visual Studio Code Extensions

Visual Studio Code Extensions(Click to Expand)

Extension	Author	Version
bracket-pair-colorizer-2	CoenraadS	0.1.4
code-settings-sync	Shan	3.4.3
github-linker	gimenete	0.2.3
gitlens	eamodio	10.2.1
grammarly	znck	0.12.1
markdown-all-in-one	yzhang	3.0.0
material-icon-theme	PKief	4.1.0
powershell-preview	ms-vscode	2020.6.0
project-manager	alefragnani	11.0.1
vscode-markdownlint	DavidAnson	0.36.0
vscode-pull-request-github	GitHub	0.16.0
vscode-toggle-quotes	BriteSnow	0.3.2
vscode-wakatime	WakaTime	4.0.0
vscode-yaml	redhat	0.8.0
xml	DotJoshJohnson	2.5.0

[TylerLeonhardt]

I found an instance in the logs of a completion request taking 35seconds (for future self, open the log file in vscode and do a regex search: \d\d\d\d\dms):

2020-06-06 15:27:34.289 +02:00 [VRB] Request 514 was abandoned due to content be modified
2020-06-06 15:27:34.289 +02:00 [DBG] Completion request canceled for file: file:///c:/Support/GitHub/Testimo/Private/SourcesDomain/GroupPolicyADM.ps1
2020-06-06 15:27:34.290 +02:00 [DBG] Finished: Routing Request (514) textDocument/completion in 35591ms
2020-06-06 15:27:34.290 +02:00 [VRB] Request 514 was cancelled
2020-06-06 15:27:34.290 +02:00 [DBG] Finished: Processing request textDocument/completion 514 in 35591ms

But I'm not sure if it was stuck in PowerShell or not. Either way, I think completion should be a cancellable action... I'm hoping PowerShell itself will handle it gracefully... so I think we should dupe this to PowerShell/PowerShellEditorServices#1295

because I want a requirement if that work to support cancellation.

Let me discuss this in Triage tomorrow before I mark it.

[rkeithhill]

Or \d{5,}ms. :-)

[mattpwhite]

I can confirm that 2020.6.0 is better, but still falls over regularly. The extension logs show requests for completions just hanging until I give up after 20+ seconds (at which point they are "cancelled", which I believe is a lie in the current implementation). This is different than the behavior seen in earlier versions in that the extension itself isn't quite toast (in those cases I'd just see the log basically stop updating), but the user experience is still similar. I still have to kill the extension all the time to get it back into a working state.

I'll try to collect some useful logs later today.

[mattpwhite]

Here's an interesting snippet from a couple minutes ago:

• Does a command lookup for a function defined in the file I'm currently editing. That fails.
• An exception with a very long async stack trace about concurrent access to the pipeline
• A 52 second hang (request 8064) on a textDocument/definition call right after that, during which various extension features like folding were not working.

pses-excerpt.log

My log is pretty rotten with sequences like that. Also a fair number of these exceptions about entering nested pipelines.

other-exceptions.txt

[rjmholt]

Thanks for looking into that @mattpwhite! Looks like further motivation for PowerShell/PowerShellEditorServices#1295

[PrzemyslawKlys]

1596305359-91a018eb-cd5f-4159-a261-8ba50c0e8b2f1596305357826.zip

Another log. Basically after first cancelled event everything goes to hell. It can go to hell as quick as 5-20 minutes in.

[raandree]

The last years I have always moved to the ISE when writing more then 5 lines of code in PowerShell as IntelliSense is still not reliable at all. This week I am developing for Universal Dashboard and tried with VSCode as there is a plugin for UD. It is extremely painful. If the script gets a bit bigger, IntelliSense does not work at all. Closing the terminal does not solve the issue. It feels like in the old VBS times.

Another area is DSC. When writing configurations or interacting with DSC cmdlets, IntelliSense dies.

What data would help working on this?

[SydneyhSmith]

We recently published an update to our preview extension which has addressed a number of issues, would you mind giving it a try and letting us know if you are still hitting this?

[SydneyhSmith]

@raandree when you reference the script getting bigger, what size of script are you roughly seeing?

[raandree]

@SydneyhSmith, right after commenting on this issue I got the notification about the new version of the PowerShell Preview extension. It feels much better!

One issue is still there. IntelliSense works in the terminal but not in the code editor. I have created a screen recording of the behavior: https://1drv.ms/v/s!AuCqmYWYQ_kMnohF-j93qXWJsmtBXA?e=SLid98

[SydneyhSmith]

Thanks @raandree for the screengrab that is super helpful! Noticed the red squigglies when you are expecting the completions---this may be due to the placement of the the cmd not being valid so we wouldnt expect completions to work there...what is the squiggly you are getting?

[tmknight]

I am still seeing regularly seeing this in both latest (as of this message) stable and preview of PowerShell Integrated Console

This is expected:

Version: 1.49.2 (user setup)
Commit: e5e9e69aed6e1984f7499b7af85b3d05f9a6883a
Date: 2020-09-24T16:29:41.983Z
Electron: 9.2.1
Chrome: 83.0.4103.122
Node.js: 12.14.1
V8: 8.3.110.13-electron.0
OS: Windows_NT x64 10.0.19041

[PrzemyslawKlys]

I just want to add that for me, my problems happen on minutes/hourly basis. It's not occasional anymore. It may be related to my other issue #2908 but generally, it's pretty bad. I either stop using IntelliSense and try to pick things from memory or I restart session all the time. I can only imagine how it works for other people that never report issues thinking code just sucks.

[danstur]

@SydneyhSmith I'm still experiencing this issue pretty regularly. Do you need extra logs to analyse the issue or what's the reason the Needs: Attention flag was removed?

If there's a way I can help get to the root of this issue, I'd be more than willing to help.

[PrzemyslawKlys]

But the new Preview version is much younger than several months. Are you sure it's a new preview and could you retest with the current preview just to make sure?

[JustinGrote]

There are extended logs but they may not cover just simple completion services, especially just "hangs". You may want to try making a "fresh" portable vscode installation so you don't have to affect your existing setup, just download the zip of vscode and add a "data" folder to the extract.

[SeeminglyScience]

That doesn't seem to help. Btw we use VSC to manage our company PowerShell GIT repository which contains approximately 3000 files (1000 of them are ps1) and 800 directories, so maybe it is somehow related to its size?

Yeah that'll cause some issues for sure. Try adding this to your settings.json

"[powershell]": {
    "editor.codeLens": false,
},

[ztrhgf]

That doesn't seem to help. Btw we use VSC to manage our company PowerShell GIT repository which contains approximately 3000 files (1000 of them are ps1) and 800 directories, so maybe it is somehow related to its size?

Yeah that'll cause some issues for sure. Try adding this to your settings.json

"[powershell]": {
    "editor.codeLens": false,
},

This seems to help, but to be sure I will test it for a longer period. Thanks anyway!

[SeeminglyScience]

@andschwa just a thought, we might want to look at using some heuristics to turn that off automatically in some workspaces. Ideally with an option to force it on always.

As much as I'd like to guide folks to smaller workspaces, an very high number of folks just want to open up all their stuff at the same time. To the point where if someone is talking about issues with intellisense on discord, half the time I open with "how big is the workspace" and the answer is often "all of the stuff".

[ztrhgf]

(from my point of view) would be better to allow define folders that should be searched. Or some sort of customizable solution. Because it is very user friendly to have all your organization code at one place and be able to search it at once etc. Thanks in advance Sebela

[JustinGrote]

@ztrhgf FYI that sparse checkout is also a thing:
https://git-scm.com/docs/git-sparse-checkout

[ztrhgf]

@ztrhgf FYI that sparse checkout is also a thing: https://git-scm.com/docs/git-sparse-checkout

sorry I don't understand what you mean by that

[JustinGrote]

@ztrhgf you can check out only a portion of a git repository (say an individual folder) without checking out the whole thing, similarly you could also "open folder" in vscode on that particular folder to cut down on the amount of files Powershell has to process.

That is just a workaround in the meantime, of course it's a viable ask but it may take some time to implement.

[ztrhgf]

I see. Unfortunately, it is of no use for us because we have a custom CICD solution that requires all data to be locally available. But thanks for the tip anyway!

I will test this further, but so far it seems that setting

"[powershell]": {
    "editor.codeLens": false,
}

when combined with the Preview version of the PowerShell extension helped a lot. It is definitely not perfect but is usable.

[andyleejordan]

Related featured request would make this configurable and probably help: #3884

[andyleejordan]

@PrzemyslawKlys should we close this issue or are you still seeing the same thing?

[PrzemyslawKlys]

@andschwa We can close it because in my cases I believe it's related to [Types] stopping working and I believe @SeeminglyScience once explained it's a problem on PS 5.1. So I just try to use PS 7 most of the time.

I am not sure whether you want that one as part of "issues", something that would be considered for a fix in a workaround in VSCode extension? Also I've also mentioned my problems here: #2745 (comment) but those are edge cases and I guess it would be again workaround in VSCode extension rather than anything else. Do you want them as issue/issues or should we just assume those are non-fixable and we should stop wasting time on that?

[andyleejordan]

@PrzemyslawKlys I'm looking at the screenshots in your linked comment...for the first example:

function Set-Test {
    [cmdletBinding()]
    param(
        [string] $Test,
        [UnknownType] $Test1
    )
}

Set-Test -

If I understand correctly you're expecting at least -Test to show up under parameter hints, but it doesn't. And it doesn't because the bound parameters are broken since $Test1 has a type constraint of UnknownType. Removing that (or changing the type to something real, like object) makes parameter hints work just fine. So yeah, I think that's less an edge and more an "expected failure" since the code it's parsing is broken. Would I like PSSA to throw a problem at [UnknownType]? Totally, but oh well. @SeeminglyScience is there something better we can do there?

For the second example with New-HTMLGroupDiagramSummaryHierarchical I think you already hit the nail on the head: PowerShell isn't giving us parameter hints for it, so there's not much we can do there. Unless @SeeminglyScience knows a brilliant solution 😉

For the third example...I have no idea why that isn't working. I went to test it and expected it to work in the preview. Ok can you please file a separate issue about "Parameter hints are broken"? I was looking at them a little bit while improving the symbols, and realized they are very oddly implemented (for instance, we only look at parameter sets, which leads to weird suggestions) and clearly there's an issue where it's not getting any parameters for a function defined elsewhere in the workspace. It should, and I don't like that it's not!

[PrzemyslawKlys]

The idea around the first example is that I sometimes develop something without a library. For instance working on Exchange 2010/2013 script where you need to connect to Exchange to get cmdlets available, and that means if you don't have them connected/at hand, developing for it will be a nightmare. So it's not about UnknownType, it's about type that developer is not having at the moment. Sure PSScriptAnalyzer could help in detecting typos in "types" (is there such a rule?), but it's not about thatin that case.

I guess with Github CoPilot things are a bit easier ... so maybe the effort is not worth it.

The 2nd and 3rd examples are basically the same thing - the functions are stored in separate files, often in a Private folder. If you try to call them up outside of those files they will not work (in both cases). If you call them up at least once they will be in the session and work, but any changes to them won't be reflected. I would say this is how PowerShell works in VSCode now and there's no difference between my 2nd and 3rd example.

So unless you're willing to Prescan all folders somehow and provide parameters hints even for private functions (after all you're not reading PSD1/PSM1 to deliver information) it won't work.

My projects are always done the same way Private, Public folders, one function per file, and then all that is loaded up thru PSM1 and PSD1. In PSM1 during development, I have Export-ModuleMember -Function '*' -Alias '*', but in PSD1, during development, my FunctionsToExport is always set to specific values. Sure enough, I could change it to "*" and once you load module all private/public functions will be available within session so parameters and everything else will work.

So, in my opinion, the current implementation "works as expected", according to the rules of PowerShell (where files are their own boundaries), but what would be cool from sysadmin/dev perspective that spends lots of time writing modules is the ability to "switch" between standard behavior and developer behavior somehow. Where by Developer behavior I mean working cross-files (within single workspace) regardless if those are loaded to specific session or not. After all you already have References up and running so extension is aware of functions existence. This would be helpful for developers to get a bit of that "C#" magic that they get. I understand this may not be your target audience to help those very few "devs" like me, who like to separate every function into their own existence.

So I don't believe opening an issue "Parameter hints are broken" makes sense unless you're thinking @andschwa on going the extra mile on this because I believe those are as "expected" regardless of my feelings :-)

[andyleejordan]

Ok fair! One day it would be nice to go that extra mile, because honestly I agree with you. As a developer, I'd expect these to just work. But at least know why they aren't and it's "expected." Thanks!

[JustinGrote]

@PrzemyslawKlys as a non-optimal workaround, I have an alias in my profile that basically dot sources everything in my project, and if you do that you will get the intellisense.

[PrzemyslawKlys]

@JustinGrote I mostly switch to modifying PSD1 to * and just rerunning module and it works, or just take a bullet. I have plans to learn on how to create VSCode extensions and just create some GUI for actions I do frequently such as Importing module, maybe "live" edit PSD1 and rerun, things like that. Something that will make my development a bit less painful in some situations. I don't know yet when I will be able to do it - but I have been thinking on it for a while.

Dot sourcing for hybrid modules is usually not enough.