Bug 709 - MD intellisense beachballs the machine
Summary: MD intellisense beachballs the machine
Status: RESOLVED FIXED
Alias: None
Product: iOS
Classification: Xamarin
Component: General ()
Version: 4.x
Hardware: Macintosh Mac OS
: --- blocker
Target Milestone: Untriaged
Assignee: Bugzilla
URL:
: 803 ()
Depends on:
Blocks:
 
Reported: 2011-09-10 06:52 UTC by Nic Wise
Modified: 2011-09-19 07:37 UTC (History)
8 users (show)

Tags:
Is this bug a regression?: ---
Last known good build:


Attachments
debug log from a kill -QUIT (70.51 KB, text/plain)
2011-09-10 17:27 UTC, Nic Wise
Details
MonoTouch-lib which can't be listed with unzip (665.50 KB, application/octet-stream)
2011-09-14 02:52 UTC, Nic Wise
Details
MD 2.6 with MT 4.1.1.1 log file from around comment #34 (56.11 KB, application/octet-stream)
2011-09-14 17:03 UTC, Nic Wise
Details
MD 2.6 with MT 4.2 log file from around comment #34 (65.35 KB, application/octet-stream)
2011-09-14 17:03 UTC, Nic Wise
Details
Fixed MonoTouch-lib.zip (2.14 MB, application/zip)
2011-09-14 17:45 UTC, Jeffrey Stedfast
Details


Notice (2018-05-24): bugzilla.xamarin.com is now in read-only mode.

Please join us on Visual Studio Developer Community and in the Xamarin and Mono organizations on GitHub to continue tracking issues. Bugzilla will remain available for reference in read-only mode. We will continue to work on open Bugzilla bugs, copy them to the new locations as needed for follow-up, and add the new items under Related Links.

Our sincere thanks to everyone who has contributed on this bug tracker over the years. Thanks also for your understanding as we make these adjustments and improvements for the future.


Please create a new report on Developer Community or GitHub with your current version information, steps to reproduce, and relevant error messages or log files if you are hitting an issue that looks similar to this resolved bug and you do not yet see a matching new report.

Related Links:
Status:
RESOLVED FIXED

Description Nic Wise 2011-09-10 06:52:58 UTC
The subject line is dramatic for a reason, and is quite a good summary.

My setup:
MBP i5 2.6 (dual core), 8GB, Seagate hybrid SSD. No other major apps running. Snow Leopard
Mono 2.10.5
MD 2.6 RTM
MT 4.2, or 4.1.1.1
XCODE 3.something (latest 3.x release), SDK 4.3 (latest non-5.0 beta)

Since 4.1.1.1, intellisense has been unusable, which makes MonoDevelop unusable without turning IntelliSense off. (I was about to try it without IntelliSense, but I can't find out how to do that!)

As it's hard to explain just how slow it is, I've made a short video/screencap of it happening. This happens on all MT projects I have, but if I make a new Console app, it does NOT do it. IntelliSense works fine. Hence I think it's an MT bug, not a MD bug.

the video is here: http://www.fastchicken.co.nz/stuff/md26bug.mov - 200meg. (available after 1130 GMT / 1230 BST / 0430 PST, as it's still uploading!)

Sadly, you can't see the mouse cursor, but if you watch the cpu graph in to menu bar (next to the padlock) it gives you an idea of whats going on.

Is there any way for me to provide more debug info? I'm also happy to run test builds of MT/MD, or give someone from Xamarin remote access to my laptop if you need it.
Comment 1 Sebastien Pouliot 2011-09-10 10:28:17 UTC
Switching to MonoDevelop since there's nothing from MonoTouch itself *running* when you edit code. However code-completion does have to read all MT assemblies so that could be the interaction, from MT, that cause the issue.

FWIW I did not have this issue with:
iMac i5 (quad core), 12GB, 500GB, lots of other apps running
Snow Leopard
Mono 2.10.5
MD 2.6 RTM
MT 4.2 (or close, i.e. 4-2 branch)
XCode 4 (did not really use it with MD 2.6)
SDK 4.3
Comment 2 Nic Wise 2011-09-10 10:53:27 UTC
Is there any way to get a debug output or something from MD? eg -E:all etc and then upload the output?
Comment 3 Mikayla Hutchinson [MSFT] 2011-09-10 16:13:44 UTC
While beachballed, you could try to get a stack trace: http://monodevelop.com/Developers/Reporting_Bugs#Managed_Hangs
Comment 4 Nic Wise 2011-09-10 17:26:51 UTC
Done. I did the kill about 4 times while it was hung. Attached to this.
Comment 5 Nic Wise 2011-09-10 17:27:32 UTC
Created attachment 306 [details]
debug log from a kill -QUIT

results from 4 kill -QUIT's while it was hanging
Comment 6 Mikayla Hutchinson [MSFT] 2011-09-10 18:58:51 UTC
Thanks, that narrows it down. It looks like it's blocking on the disk when trying to read the documentation summaries. Don't know why that's a problem for this particular help file though.
Comment 7 Nic Wise 2011-09-11 06:58:19 UTC
Is there anything I can do / remove / clean up and reinstall? eg, uninstall MT and reinstall it? Not even sure how I can do that.

I did just try to use disk util to repair perm's, but while it found a load of stuff, nothing was from Mono/MD/MT.
Comment 8 Mike Krüger 2011-09-12 01:44:20 UTC
Reinstall may help - this stack trace should never happen a block in read seems to be an underlying problem with the files.
Comment 9 Nic Wise 2011-09-12 04:10:15 UTC
I reinstalled 4.2, but it made very little difference. it still takes between (at a guess) 1s and 3 seconds for the intellisense to popup (and thats blocking all the time)

Is there a way to:

a) disable intellisense?

b) uninstall MT totally, so I can reinstall it?

the reinstall of MT took a while - the first bit goes quickly, but once it's running package scripts, it takes maybe 90 seconds to do the rest, of which mono is using 100% cpu (well, 99%, so 50% each of 2 cores)

can I switch back to Mono 2.10.4 to see if that changes it? If so, how? I have it installed, just not sure how to switch back.
Comment 10 Nic Wise 2011-09-13 05:00:09 UTC
**ping**

any ideas on how I uninstall MT cleanly or disable intellisense?
Comment 11 Mike Krüger 2011-09-13 05:54:25 UTC
Intellisense can be disabled in the text editor options - general tab the 1st checkbox.

The problem may be the monodoc zip files that takes ages to unzip - we need to do something about that.
Comment 12 Nic Wise 2011-09-13 07:33:21 UTC
Why is no one else seeing it? It's not like my machine is under powered or even under load.

MBP 15" i5 2.6ghz, 8GB ram, Momentus XT disk (so about 5x quicker than a normal hdd)

And, why is it being opened and constantly re-read each time? wouldn't that be done once and held in RAM? And why only since 4.1.1.1 - it was fine before that.
Comment 13 Mikayla Hutchinson [MSFT] 2011-09-13 07:50:19 UTC
My guess would be that the Momentus' adaptive file caching is doing something strange, and affecting that file purely by chance.

In general, however, I can't say I'm comfortable with reading the doc files in the completion engine. Maybe we should build doc caches like VS does?
Comment 14 Mike Krüger 2011-09-13 07:51:58 UTC
Y I already thought about just putting the monodoc descriptions into the code completion DB. 

I think this is the issue people complain about when they've poor performance.
Comment 15 Mikayla Hutchinson [MSFT] 2011-09-13 08:00:28 UTC
IIRC we used to do that, and the issue was that if docs were localized and the locale changed, the completion would still use the docs of the old locale. But I suppose we could just localize the whole completion DB - either have multiple variants (e.g. Foo.en.pidb), or embed the locale name into the db and regenerate it if the locale changes.
Comment 16 Mike Krüger 2011-09-13 08:37:36 UTC
Theoretically yes - but do we have localized mono docs ?

I did put the monodoc documentation in the code completion database - that should solve the speed problems.

btw. The Foo.en.pidb approach is the easiest one I think.
Comment 17 Nic Wise 2011-09-13 10:15:12 UTC
I doubt it's the Momentus XT. If it was, I'd expect a reinstall to "fix" it.

Still: how do I remove all this stuff, and (try to) reinstall it? just delete /Developer/MonoTouch?
Comment 18 Nic Wise 2011-09-13 10:16:01 UTC
I should clarify that, at the moment, I can't use MT/MD. Which is a wee bit frustrating. OK, I can, but after about 5 mins, I want to throw my laptop thru a window, so I'll go with "can't" for the moment.
Comment 19 Mikayla Hutchinson [MSFT] 2011-09-13 14:19:44 UTC
Mike, XML doc comments can be localized, and we do support them.
Comment 20 Mike Krüger 2011-09-14 01:27:40 UTC
it's not about xml docs it's about monodocs ... xml docs are a different thing and handled elsewhere.
Comment 21 Mike Krüger 2011-09-14 01:33:06 UTC
@Nic Wise: I suspect that someone used a stronger compression / other compression algortihm for the montouch monodocs - this one seems to be problematic for the #ziplib.

(I haven't touched #ziplib for years)

Therefore it's more a packaging problem why code completion is slow.
Comment 22 Mike Krüger 2011-09-14 01:50:37 UTC
@Nic Wise:

Can you try doing this - this repackages the MonoTouch-lib.zip with a low compression level.

mkdir /tmp/MonoTouch-lib
unzip /Developer/MonoTouch/usr/share/doc/MonoTouch/MonoTouch-lib.zip -d /tmp/MonoTouch-lib
sudo zip -0 /Developer/MonoTouch/usr/share/doc/MonoTouch/MonoTouch-lib.zip /tmp/MonoTouch-lib/*
rm -rf /tmp/MonoTouch-lib
Comment 23 Nic Wise 2011-09-14 02:46:54 UTC
Thanks @Mike

I think that showed up the problem


hypermania:MonoTouch nic$ unzip -l MonoTouch-lib.zip
Archive:  MonoTouch-lib.zip
  End-of-central-directory signature not found.  Either this file is not
  a zipfile, or it constitutes one disk of a multi-part archive.  In the
  latter case the central directory and zipfile comment will be found on
  the last disk(s) of this archive.
unzip:  cannot find zipfile directory in one of MonoTouch-lib.zip or
        MonoTouch-lib.zip.zip, and cannot find MonoTouch-lib.zip.ZIP, period.
hypermania:MonoTouch nic$ 


So, I have a corrupt zip file. I might delete it and reinstall. Or can you attach a known-working one to this?
Comment 24 Nic Wise 2011-09-14 02:51:04 UTC
OK, deleted /Developer/MonoTouch and reinstalled. No change.

I've attached the zip file to this.
Comment 25 Nic Wise 2011-09-14 02:52:15 UTC
Created attachment 358 [details]
MonoTouch-lib which can't be listed with unzip

Attached. This is from a clean (delete then reinstall) of MT4.2
Comment 26 Nic Wise 2011-09-14 02:52:51 UTC
and if it's any help. MonoTouch-man.zip (same folder) works fine.
Comment 27 Nic Wise 2011-09-14 04:09:41 UTC
ok, so I used zip -FF to fix it

sudo zip -FF MonoTouch-lib.zip --out MT-lib.zip
sudo rm MonoTouch-lib.zip
sudo mv MT-lib.zip MonoTouch-lib.zip

which "fixed" it (I can zip -l on it now) however MD is still slow for intellisense. I even removed the zip file (renamed it) and it's still slow - just without MT content :)
Comment 28 Wei-Yuen Tan 2011-09-14 04:13:58 UTC
Same problem here - glacial autocomplete/intellisense renders IDE unuseable.  Environment:

2010 Mac Mini 4GB, no other apps running
Snow Leopard
Mono 2.10.5
MD 2.6 RTM
MT 4.2 
XCode 3
SDK 4.3
Comment 29 Mike Krüger 2011-09-14 06:18:15 UTC
@Nic WIse: if md ist still slow can I have another kill -QUIT for it ?

It may be that there are some files missing and the monodoc provider slows things down. MD now doesn't use monotouch docs anymore - it stores them in the code completion db which works a bit faster - but it shouldn't be noticeable on user level.

Therfore more kill -quit outputs may help.
Comment 30 Nic Wise 2011-09-14 06:21:13 UTC
Sure thing. I'll do it when I get home tonight (6pm or so, GMT).
Comment 31 Jeffrey Stedfast 2011-09-14 16:27:40 UTC
Confirmed that the MonoTouch-4.2 MonoTouch-lib.zip is corrupted with a fresh install.
Comment 32 Sebastien Pouliot 2011-09-14 16:39:47 UTC
Mike already opened bug #800 on MonoTouch about the corrupted zip file.

However comment #26 shows it still slow (with fixed zip file) on MD and that it's under investigation (#29) awaiting more user data (#30) so I'm giving you back this one ;-)
Comment 33 Jeffrey Stedfast 2011-09-14 16:42:12 UTC
[fejj@localhost MonoTouch]$ file MonoTouch-lib.zip 
MonoTouch-lib.zip: Zip archive data, at least v2.0 to extract
[fejj@localhost MonoTouch]$ ls -l MonoTouch-lib.zip 
-rw-r--r--  1 fejj  admin  456602 Sep 14 16:19 MonoTouch-lib.zip

Is Zip 2.0 some format that MonoDevelop (and MacOSX 10.6.8's unzip command) can't extract?

After talking with Alex (our packaging guy), it sounds like the MonoTouch-lib.zip file is generated by the post-install scripts using mdoc (which might explain why it works for some people and not others).

Somehow there must be something going wrong depending on what versions of Mono* software they have installed.
Comment 34 Jeffrey Stedfast 2011-09-14 16:43:50 UTC
FWIW:

mdoc --version
mdoc 2.10.5.0
Comment 35 Nic Wise 2011-09-14 17:01:49 UTC
ok, I see Jeff has posted that it's 4.2, but here's some more info, and I'll do a kill -QUIT with 4.2 and the others and attach them after this

Mono 2.10.5 fresh download and install
MD 2.6
MT 4.1 (glad I save the dmgs!)

Works fine. No issues. Installs (the package scripts bit) in about 10 seconds.

4.1.1.1 install took a LOT LOT longer in the running package scripts. like 10 seconds vrs 2 mins just for that step. Mono is pegging one core (well, 50% *2) for the whole of that 2mins.

Same problem as always. See MD26-MT4111.log
doc zip file is correct. Filesize is 681470

4.2 install:
Same as 4.1.1.1 - takes ages to install.

This shows in the console (tho we know about that)
14/09/2011 21:51:30	/System/Library/Frameworks/CoreServices.framework/Frameworks/Metadata.framework/Versions/A/Support/mdworker[1120]	zip importer encountered an error (2) scanning "/Developer/MonoTouch/usr/share/doc/MonoTouch/MonoTouch-lib.zip".
14/09/2011 21:51:31	/System/Library/Frameworks/CoreServices.framework/Frameworks/Metadata.framework/Versions/A/Support/mdworker[1120]	zip importer encountered an error (2) scanning "/Developer/MonoTouch/usr/share/doc/MonoTouch/MonoTouch-lib.zip".
14/09/2011 21:51:31	/System/Library/Frameworks/CoreServices.framework/Frameworks/Metadata.framework/Versions/A/Support/mdworker[1120]	zip importer encountered an error (2) scanning "/Library/Frameworks/Mono.framework/External/monodoc/MonoTouch-lib.zip".

.zip file size is the same.

Unsurprisingly 4.2 has the same problems.

log files coming shortly.
Comment 36 Nic Wise 2011-09-14 17:03:19 UTC
Created attachment 371 [details]
MD 2.6 with MT 4.1.1.1 log file from around comment #34
Comment 37 Nic Wise 2011-09-14 17:03:47 UTC
Created attachment 372 [details]
MD 2.6 with MT 4.2 log file from around comment #34
Comment 38 Nic Wise 2011-09-14 17:08:09 UTC
OK, log files up.

My suspicion is that SharpZipLib is looking for the directory (end of the zip) which doesn't exist. it fails, so it scans the whole file (as the file info is in the file too - ie:

zip header
file header
file data
file header
file data
...
directory (which is missing)

and it eventually finds the stuff, but only after unzipping the whole file.

And then, when I type another key, it does it again.

Make sense?

@Jeff, here's my file info, ala your output in #34

hypermania:MonoTouch nic$ file MonoTouch-lib.zip
MonoTouch-lib.zip: Zip archive data, at least v2.0 to extract
hypermania:MonoTouch nic$ ls -l MonoTouch-lib.zip
-rw-r--r--  1 root  admin  681470 14 Sep 21:51 MonoTouch-lib.zip
hypermania:MonoTouch nic$ mdoc --version
mdoc 2.10.5.0
hypermania:MonoTouch nic$

I think file is looking at the header, ie "PK", which is a zip, but not actually validating the zip file itself. If I less it, I can see that it's missing the directory footer at the end (I wrote a zip file app in about 1990, so I'm semi-familiar with the file structure)
Comment 39 Jeffrey Stedfast 2011-09-14 17:35:36 UTC
I've got a fix! Whoo! 

So the problem was my poor xml skills when documenting some APIs (see! this is what I get for documenting stuff!!). I had forgotten to close an xml tag so mdoc was outputting a garbage .zip.

As it happens, I fixed this ages ago but my fix must have just barely missed the 4.2 branch so I simply needed to cherry-pick it over into the 4.2 branch and now the docs build fine.

I'll try to get Alex to build a new MonoTouch-4.2.1 tomorrow with my fix.

Thanks for your patience and help, Nic!
Comment 40 Nic Wise 2011-09-14 17:39:59 UTC
woohoo :) I'll give the update a go tomorrow then!

I'm surprised more people haven't hit the problem tho. Too many SSD's out there?

If he can't, just fire over the zip file ;-)

On the plus side, I now know a lot more about MT's internals (Well, kinda) than I did before.
Comment 41 Jeffrey Stedfast 2011-09-14 17:44:07 UTC
Nic: I let Alex know he needs to roll a 4.2.1 release so he's on the case. In the meantime, if you'd like me to email you my fixed MonoTouch-lib.zip (or attach it here to bugzilla), I can do that.

It's about 2.2 MB - just let me know.
Comment 42 Jeffrey Stedfast 2011-09-14 17:45:44 UTC
Created attachment 373 [details]
Fixed MonoTouch-lib.zip

Attached my freshly created MonoTouch-lib.zip which zip can read just fine. Yay!
Comment 43 Wei-Yuen Tan 2011-09-14 17:46:00 UTC
Jeffrey, Please do so -- I want it too.  Thanks.
Comment 44 Wei-Yuen Tan 2011-09-14 18:09:17 UTC
As I understand it, if I replace /Developer/MonoTouch/usr/share/doc/MonoTouch/MonoTouch-lib.zip with the new zip file that you attached, all should be fixed, right?

If so, this does not seem to resolve anything for me.  Autocomplete/Intellisense still agonizingly slow.
Comment 45 Jeffrey Stedfast 2011-09-14 19:05:21 UTC
Hmmm, try copying it to /Library/Frameworks/Mono.framework/External/monodoc/ as well (for some reason we are installing the same zip into 2 locations)
Comment 46 Wei-Yuen Tan 2011-09-14 19:12:02 UTC
Tried copying to /Library/Frameworks/Mono.framework/External/monodoc/

No improvement.  Nic, does it make any difference for you?
Comment 48 Jeffrey Stedfast 2011-09-14 19:18:31 UTC
Wei-Yuen: are you able to unzip my zip ok? Maybe it didn't download properly?

What if you just delete /Developer/MonoTouch/usr/share/doc/MonoTouch/MonoTouch-lib.zip and /Library/Frameworks/Mono.framework/External/monodoc/MonoTouch-lib.zip?

Does MonoDevelop still have the same issue?
Comment 49 Wei-Yuen Tan 2011-09-14 19:30:40 UTC
Actually, I think I'm seeing some improvement now, although some things (e.g. typing "this.") are still very slow.  Will report back with more detail as I go.

Thanks.
Comment 50 Wei-Yuen Tan 2011-09-14 19:38:01 UTC
Still very slow if I type anything on a new line (e.g. declare a new variable).  Lags for 3+ seconds before I can declare just a simple string or integer.
Comment 51 Mike Krüger 2011-09-15 02:19:17 UTC
@Nic: does the new zip work for you, if not it would be nice if you can get me another stack trace. The last one still points to a problematic zip file.

Maybe there is another one in the mono installation that causes this trouble :(

(I reworked that layer on 2.8, but if the zip file is problematic the creation of the code completion database >might< never finish - which isn't very good as well)
Comment 52 Nic Wise 2011-09-15 02:41:32 UTC
@Mike

I put the file in both places, and it has 100% fixed the problem. Thanks so much, everyone :)

FWIW, "this", "var" etc all work fine, too. It's back to how it was with 4.1
Comment 53 Mike Krüger 2011-09-15 03:01:28 UTC
*** Bug 803 has been marked as a duplicate of this bug. ***
Comment 54 Wei-Yuen Tan 2011-09-15 03:02:46 UTC
Unfortunately it still doesn't work for me.
Comment 55 Nic Wise 2011-09-15 03:20:40 UTC
Wei, you might need to do the kill -QUIT thing which is detailed in #3 and post the log file (you do a save as from the Console.app
Comment 56 Wei-Yuen Tan 2011-09-15 03:49:49 UTC
OK, it looks like it made a difference.  Intellisense is working again...mostly.  It is fast and it doesn't block input.

HOWEVER...what I'm finding is that it doesn't work on references to classes defined in the current solution, whereas this was working before.
Comment 57 Wei-Yuen Tan 2011-09-15 04:04:17 UTC
Given the following code:

	public class bar {
		public string s1 {get;set;}
		public string s2 {get;set;}
		
		public bar () {
			s1 = string.Empty;
			s2 = string.Empty;
		}
	}
	
	public class foo {
		public bar b1;
		
		public foo () {
		}
		
		public void test() {
			b1 = new bar();
			Console.WriteLine(b1.s1);
		}
	}

In the call to Console.WriteLine (last line of code before the closing braces), Intellisense is not working when I type "b1.s1"...none of the members of bar are suggested.
Comment 58 Wei-Yuen Tan 2011-09-15 04:05:06 UTC
Oh and BTW...what got the Intellisense half working was a restart of my machine after copying the Monotouch-lib.zip files to the two locations.  It did not start working again until after a restart -- why?
Comment 59 Mike Krüger 2011-09-19 07:37:32 UTC
it's fixed on the monodevelop side. 

2.8 will have much better handling of invalid montouch ZIP files. The cause:

http://bugzilla.xamarin.com/show_bug.cgi?id=800

is fixed as well.