Tk Source Code

The issue resembles the one reported in [88f02fa8]: after starting Tk (e.g. running wish or doing package require Tk from tclsh), the menu bar will be unresponsive. Deactivating (e.g. switch to some other app like Finder) and reactivating Tk will make the menu bar responsive.

I observe this issue both with 8.6.9.1 and with recent core-8-6-branch. I used Tk built with Xcode 11 beta 5 command line tools, on macOS 10.15 Catalina public beta 5 4 (19A526h). I do not observe this issue in 10.14 Mojave.

I dusted off a Mac and have confirmed that bug-bf93d098d7-redux [5bb1439eeb] finally fixes the issue for me. Thank you Marc for your effort on this, and thanks to everyone who reported their findings.

Thanks everyone for testing.  The bug-bf93d098d7-redux has been merged to
core-8-6-branch and trunk.  I am closing this ticket now.

@bll:
 
not sure what I should report exactly: in fact, the macports-installed 8.6.10 works just fine with the macports-installed tclreadline 2.3.5 here.

so it is only the self-compiled 8.6.10 which does segfault...

the first question thus seems to be: what is the difference between 8.6.10 from macports and 8.6.10 self-compiled?

@veedeeh:

You should open a new ticket for the readline crash.
Open a ticket at the tclreadline sourceforge project also.

The tclreadline from MacPorts crashes with 8.6.8 and 8.6.10 for me.
Works with the 8.6.9 in MacPorts.

@marc_culler:

so I now have registered and an ID (no longer anonymous...):

I also can now confirm that bug-bf93d098d7-redux fixes the menubar issue. thank you for this!

@nab:
thank you for this HowTo. I followed this, but the resulting binaries segfaulted just the same :|.  as it turns out the segfault is caused by a 

package require tclreadline

in my tclshrc. any idea why this happens (with the 8.6.10 release, too) in contrast to the tcl install via macports of the the same version?

Marc, the fix in bug-bf93d098d7-redux addresses the issue. The menu is now responsive when Tk is launched from tclsh. Thank you. Please commit.

@anonymous,
in order to compile Tcl&&Tk, I copy both .tar.gz in a specific directory, let's say myBuildDir, then :
cd myBuildDir
tar -xvzf tcl.tar.gz
cd tcl
make -C macosx deploy  CFLAGS_OPTIMIZE="-O2 -mmacosx-version-min=10.14"
sudo make -C macosx install-deploy
tar -xvzf tk.tar.gz
cd tk
make -C macosx deploy  CFLAGS_OPTIMIZE="-O2 -mmacosx-version-min=10.14"
sudo make -C macosx install-deploy

results are in /Library/Frameworks

++

@marc:

I just tried to compile (both 8.6.10 release and  tip of core-8-6-branch). I did this in the ./unix subdir: compile succeeds. starting tclsh causes segfault...

I am not running self-compiled tcl usually so I am a bit at a loss here how to proceed. any advice? and any better channel of communication than flooding the ticket history? or is this the preferred way?

OK.  I think this may *really* be fixed now.  Please test the bug-bf93d098d7-redux
branch.  It is so helpful to have a simple, reliable way of demonstrating the bug.

Of course we should fix this if we can figure out how to.

It might help to agree on the symptoms.  I definitely do not see this when I
type wish in a shell window.  But I do see it when I run tclsh in the shell and
then run the command package require Tk.  It would help if @anonymous would
try using the tip of the core-8-6-branch instead of macPorts. In fact today
was the first time that I have seen this behavior myself; I did not know it
was that simple.  What I heard earlier is that it happens in a program which
is much too complicated to allow for producing a simple demonstration.  And
that program was later fixed by removing some mysterious lines from the
info.plist file.

I am pretty sure that the underlying problem is a race condition in the app
initialization, but simple things like adding delays do not seem to fix it.
Remember that we are flying blind here.  Tk is forced to replicate Apple's
undocumented proprietary event loop.

One thing that I can see with my testing is that when an app is in this state
the Apple window manager never generates the NSMenuDidBeginTrackingNotification
which is normally generated when the mouse button is pressed in the menubar.
Also it is possible to restore the menu functionality programmatically by
running
% wm iconify .; wm deiconify .
but when you do that you have to watch the animation.

I would normally agree that wish is enough.

But chris has presented one use case, and apparently the python/tkinter users are also having issues.   Is  there some way to delay the appropriate initialization step slighly?

It is rather curious that chris is getting a different behaviour.

I personally have no issues with how it works now. 

My application must use wish startup for proper icons (iirc) and for proper global menu changes (even though i do not use the global menu, i do want to see my app name there) .

me again, for a third time:

@kevin_walzer

I (of course) don't know the internals but if `wish' (the command line program ) is not just a wrapper executing `tclsh' plus `package require Tk',  than it is *not* true that `wish' behaves sane and it is only tclsh+Tk that causes the problem.

of a (clickable?) GUI app `Wish' I don't know and don't use that. everything should of course work from the command line. and calling `wish' from the command line does NOT work (w.r.t. to the menubar issue) for me in the described setup just the same as tclsh+require Tk does not work.

also, I don't think you are right with "if we are talking about a GUI, then in my view Wish is the way to go". simple counter example: I do have a tcl/tk program with a CLI. something like 'myprog subcommand'.
and only _one_ subcommand (surprisingly named "view" ;)) requires Tk and pops up a graphics window. so of course my programme primarly runs as a tclsh program (and most things work w/o Tk...). I do _not_ want to make it execute as a `wish' app, naturally.

At a deeper level, here is what appears to be going on: in tk/macosx/TkMacOSXInit.c, Marc has tweaked the code to do a lot of spinning of both Tcl's event loop and Cocoa/NSApp event process to ensure that all events are firing correctly when Wish finishes launching. It works fine when Wish is launched. 

Launching Wish from tclsh seems to upend this careful sequencing of event processing to ensure smooth startup - it appears Tk is fully initialized before the NSApp events are engaged, which leads to the unresponsive menu (a symptom we've seen in the past). It seems likely that Tk has a slightly different startup process when launched from tclsh vs. Wish. 

Given that Wish works as designed (as a GUI app) on macOS, is there ever a technical reason to prefer launching Tk from a tclsh parent process instead of going to Wish directly? Even when run interactively from the Terminal, Wish runs smoothly and has all the same interactive features as tclsh. I understand running tclsh if you don't want the GUI overhead, but if we are talking about a GUI, then in my view Wish is the way to go. 

I've poked around in the code and I do not see any clear way to tweak the startup process beyond what Marc has done. Marc might be able to dig into this, but my view is pretty clear: we have a stable, fully functioning solution already. And if there's a use case other than habit/personal preference for tclsh that I'm missing, then of course please enlighten me.

My testing suggests that this applies when Wish is launched from a tclsh parent process with package require Tk, and not when launched as an app or launched *as wish* from the command line. I can't reproduce except when loading Tk as a package from tclsh. This is based on the tip of core-8-6-branch on Catalina.

me again (second to last poster): I now have confirmed the described behaviour on a third osx machine running catalina.

@bll:

as described I tested by starting `wish' from the command line.
the _exact_ same thing happens also if I start `tclsh' and issue `package require Tk'
there:

wish (aqua) window pops up and menu bar is dead until focus is switched away from wish and then back again. it's totally deterministic for me using the `macports' provided tcl/tk 8.6.10 (that should be the official release, that is).

so it definitely is *not* the case that it makes a difference whether wish is called intially or tclsh. and since it happens  with this minimal example (empty `wish' window), I obviously is not an (user) application specific thing.

me again (second to last poster): I now have confirmed the described behaviour on a third osx machine running catalina.

@bll:

as described I tested by starting `wish' from the command line.
the _exact_ same thing happens also if I start `tclsh' and issue `package require Tk'
there:

wish (aqua) window pops up and menu bar is dead until focus is switched away from wish and then back again. it's totally deterministic for me using the `macports' provided tcl/tk 8.6.10 (that should be the official release, that is).

so it definitely is *not* the case that it makes a difference whether wish is called intially or tclsh. and since it happens  with this minimal example (empty `wish' window), I obviously is not an (user) application specific thing.

I think in all cases, we need to know exactly how the program is being started.

e.g.
8.6.10: tclsh / package require Tk : menu is unresponsive
8.6.10: wish : menu OK
8.6.10: my application started as a .app : menu OK
        (uses wish).
8.6.10: my application started from the command line : menu unresponsive
        (uses tclsh).

I would guess that python/tkinter is similar to tclsh/package require Tk.

just to confirm/stress that this is a persisting and real issue: I am running catalina (OSX 10.15) and tcl/tk 8.6.10 (installed via macports) and today also hit this bug.

it is there both on a macbook pro and a mac pro. to reproduce:

* from a terminal start wish
* w/o switchting to different app (i.e `wish' displayed in left corner of menu bar the whole time) try to use menu bar. 
* result: menu bar completely dead/unresponsive, i.e. menus don't drop down
* switch to any other app via command-TAB or by clicking with the mouse
* switch back  to `wish' in the same way
* try to use menu bar
* result: behaviour back to normal

this is 100% reproducible on both machines.  I presume (not completely sure...) that I would have noted that bug in 10.14 if it had been there already. so I suspect it is a 10.15 issue.

if I can be of any help by testing patches or providing more info, let me know.

I have not used my Macs in a while, and have yet to even try a non-beta release of 10.15 Catalina or Tk 8.6.10. I unfortunately have no idea if or when I will get around to doing so.

I do not believe this issue is limited to Perl-based programs, as I had observed it by running the tclsh/wish commands (no arguments—just an interactive shell), for a clean compilation on a clean user profile on a clean install of macOS. (My use of Perl-based Tk programs was limited to running standalone Perl scripts from the command line, which are not bundled in a .app nor had their own Tcl/Tk installation.) So I believe the only plist settings involved would have been from ~/Library/Preferences/com.tcltk.wish.plist, and the only setting I've ever specified was ApplePressAndHoldEnabled for testing [92b887ca13]; no other settings (inlcuding LSUIElement) were specified or present.

I left comments on the couple of posts Takamoto found elsewhere online to let them know about this ticket in case they're interested in getting updates on or helping to investigate this issue.

@takamoto: Thank you!

This is what Apple says about that flag:

LSUIElement

A Boolean value indicating whether the app is an agent app that runs in the background and doesn't appear in the Dock.

@chrstphrchvz: By any chance, was that Boolean set to true in the app where you saw the issues with unresponsive menus?

Our Info.plist had - no idea why - "LSUIElement" set to true. If this had no apparent consequences on the demo app attached below until Mojave, on Catalina the menu was suddenly unresponsive. Simply deleting that item turns the menu fine.

@takamoto: that is great news indeed!

If you have any information about which part of info.plist was causing the
problem I hope you will share it with us.  That could save some othera a lot
of trouble!

Good news!

We have done several tests and can confirm that the menu is responsive in Catalina with the latest release, even with launching Tk with Perl.

The problem seems to be related to some mysterious change in the Info.plist specifications in Catalina.

We have tested on our hardware with Catalina, and everything seems fine. 

Thank you again

Still hoping for easy answers -- I think that all of the reports of this
problem involved perl applications.  Is it possible that this has to do with
the way perl launches a Tk app?  Are there any examples that do not involve
perl?

Thank you Marc,

long learning curve on my side since I took over from my predecessor, but I am sure this time everything is okay with the Frameworks (https://www.tcl.tk/software/tcltk/download.html).

If interested, here you can find my test app with the new frameworks integrated (app is not notarised so I am not sure it can run on Catalina at all): https://filebin.net/bsvcaa24j2884c7a 

Hiragana: this was a huge improvement! (By the way, I reported it, quote "With some delay on my side. This is a HUGE improvement. I have not been able to test it in detail, but the behaviour seems fine to me now. I will report any problem if encountered."

Thank you again for your great work.

@takamoto: Thank you for the corroboration!  Unfortunately, it doesn't apply.

The first link on your list was not related.  It was just a misunderstanding
about how menus work, and an answer was posted.

The second link has a short script, but it works fine with the current tip of
core-8-6-branch.  The menu is responsive.

The third link is about Tk 8.6.9 and also does not show the symptom with the
current tip of core-8-6-branch.

So I think the first question would be: Are you absolutely certain that you
are using the latest Tk? You have had that problem before 😉.

(That emoji is there to remind you that you never responded after the Hiragana
IME problem was fixed and the emoji palette was made to work!)

Just a short add on. The problem has been reported by many using Tcl/Tk with Python (Tkinter) on Catalina, for example:

https://stackoverflow.com/questions/58746029/tkinter-top-level-menues-not-showing-with-macos
https://stackoverflow.com/questions/58507266/menu-issue-with-python-and-catalina
https://python-forum.io/Thread-Tkinter-macOS-Catalina-and-Python-menu-issue

We are accessing Tk with Perl (Tcl::pTk) so it shouldn't be this.

I can confirm the issue in our application bundle (Tcl/Tk as Frameworks, compiled in Mojave with Xcode) and packed in .app bundle (app already in production and used on Catalina). The issue is not observable on Mojave.

A couple of observations:

As far as the bug fix proposed is concerned

    [NSApp deactivate];
    [NSApp activateIgnoringOtherApps: YES];

This - as far as I understand, but I could be completely wrong - emulates someway the reduction of the application to its icon. If this is true, even performing this manually (yellow icon of main Window) and restoring the application does not perform any change. 

The menubar can be accessed programmatically at any time.

How can I provide any help for debugging? For us it is a crucial issue.

This has now been reported on the production version of Catalina in ticket [1b819dea33].  But I still have no idea how to replicate it, so no idea
where to start debugging.

I am going to merge the bug fix branch since it seems to work fine and it does make some concrete improvement. (Scid menus work with this change.)

However, since we don't know whether the problem is completely fixed I am leaving the ticket open.

This is not something I'm concerned about, we need to hear from the OP whether
it solved his issue.

I was running commands from the command line, not from some gui: e.g.
  /Applications/VLC.app/Contents/MacOS/VLC
  /Applications/Safari.app/Contents/MacOS/Safari
and my app has:
  #!/Volumes/Users/bll/Desktop/BallroomDJ.app/Contents/MacOS/darwin/64/tcl/bin/tclsh

As long as the applications have a responsive menu when started via the icon, I think it is good.

I am not seeing the behavior you report *at all*.

One way to run a bundled app from the command line is to type the path of the executable in the MacOS directory, which is essentially what Wish does. When I run Safari that way it starts, it does not open any window, it does not post a menu bar and the app does not go into the background. Clicking the dock icon opens a window in front. Firefox and Chrome open a window in front, post the menubar and go into the background. Skim behaves like Safari. Aquamacs behaves like Firefox except it posts error messages in the terminal, does not go into the background and does not terminate when the application is killed with the Quit menu. Spotify segfaults. GIMP opens a window in the back, does not post a menubar and does not go into the background. VirtualBox opens a window in front, posts the menubar and does not go into the background. My conclusion: there is no standard behavior when you do this, and it really is not supported.

Another way is to use open -a. Most of these apps behave normally when you do this: they open a window in front, post the menubar and run in the background. But Skim does not even start while Spotify opens a window and then crashes.

If we get rid of all of the "activation" stuff then bundled Tk apps would not behave normally when their icon is clicked. That is not a good idea.

Currently we have no example of a bundled Tk app which exhibits this problem when started with open -a or by clicking the icon. And we have no example of a Tcl/Tk script which exhibits this problem when run in the shell. Since there is no standard for what should happen when the MacOS executable of a bundled app is run in the shell, I am tempted to say this problem is fixed.

No change.

I checked starting VLC and Safari from the command line.
These applications start up, appear behind my terminal window and
are not activated.  I click on them, and their menus work.

Maybe all the "activation" should just be removed???

I think I found a workaround for this. At least it fixed the one example that I had found. Please test check-in bf93d098d7 in the branch bug-bf93d098d7.

When I start my program from the command line, the menus exhibit the problem
described (on Catalina, no ideas about other Mac OS versions at this time).

If I start my program using the .app on the desktop, no issues.

The first case displays 'tclsh' as the app name, of course, and the
second case displays my application name.  Only difference I know of --
the .app has a configuration that is used.

I don't use the mac os menu, so this is not a big issue for me.

I am reopening this because I did find an app which behaves this way, namely one
of the versions of scid.  I do not see the behavior with the wish app, or with
tclsh or with IDLE or with the SnapPy app or with fossil's diff -tk.

More testing is definitely needed.

Now that I have an actual Catalina release running on real hardware I am not seeing any behavior like this.  So I am closing this ticket.

We'll have to revisit this, then, after Catalina is released, unless you want to see if you can provide a patch.

I have tested the change from catalina-menu ([3b7a3348ab]) on both Macs, and the issue persists.

I've committed the catalina-menu branch with a possible fix for this issue; can you test it on Catalina?

I have updated to the latest beta (19A546d) + command line tools and tried the latest core-8-6-branch ([16ee504b69]), but the issue still appears to be present for both of my Macs.

Commit 06623cd7ff changes the timing of events in the app startup. Since the menu problem in [88f02fa8] was also related to that timing there is a possibility that the commit had some effect on this problem. It would be worth retesting with the current tip.

I mistakenly wrote public beta 5 earlier; should say public beta 4.

I have done a clean install of Catalina public beta 4 on the 2012 MacBook Pro 13 inch, and made a clean build of Tcl and Tk from core-8-6-branch, and can still observe this issue.

I am running Catalina on a 2010 MacBook Pro 13 inch (Core 2 Duo, NVIDIA GeForce 320M), which is not supported hardware. I am using dosdude1's patcher; among what it patches is support for older, non-Metal graphics chipsets. I reported this issue anyways in case the issue appears on supported hardware. I'm not sure whether VirtualBox would necessarily be considered supported hardware either, as macOS uses EFI fallback video rather than accelerated graphics.

Maybe I can try again with a supported Mac; I have a 2012 MacBook Pro 13 inch as well.

I was not able to reproduce this on my Catalina Beta 5 system, which is running
in a VirtualBox VM on a mac mini.