Mac server; mac clients ok, windows clients get locked out

[skrying]

Here's our setup:

FileMaker server: Mac OS 10.3.9, FileMaker Server Advanced 7.0v4

Open Directory Server: Mac OS 10.4.7

FileMaker clients: Mac OS 10.3.9, Mac OS 10.4.7, Windows XP SP1 & SP2; all FileMaker Pro 7.0v3

The FM server is set to only show users the files they have access to.

Each user is a member of 1 or more groups on the OD server.

Each of these groups is set as an externally authenticated account in the relevant FM files on the FM server.

The users/groups/files are all set correctly because any user can log in to the FM server and open the files they have access to from FileMaker Pro on any Mac client.

Here's our problem: When a Windows client chooses the FileMaker Server from the list (File > Open Remote...), it immediately tries to log in to the OD server using their Windows account name and the OD server logs show "AUTH2: {some numbers, Windows account name} DIGEST-MD5 authentication failed, SASL error -13 (password incorrect)" multiple times.

This all happens BEFORE the FileMaker client prompts for a username/password to send to the server. The problem here is that if the user's Windows account name matches their OD server name, then their OD account is locked out (OD is set to lock out a user after 3 failed log in attempts), and then they can not access the FM Server. However, even if their Windows account name AND password match their account name and password for the OD server, they still get locked out. (So... if it's sending the right name and password, why is being rejected? But further, why is it sending anything at all? - the Mac client doesn't send anything at this point)

Now, if their Windows account name does not match their OD account name, or if they attempt to log in from a different Windows client, or from any Mac client, as long as their OD account isn't locked out, they can log in with no problems.

Just to clarify - the OD Server is being sent the Windows account name and password at the point when a user selects the FileMaker server from the available list (by going to File > Open Remote...) by clicking the name, but BEFORE they enter their username and password.

Questions:

- Why are the Windows clients sending the local account name and password?

- Why, if the local account name and password are valid on the OD Server, are they being rejected?

[Steven H. Blackwell]

Oh my. Is this ever a complex scenario.

First, take a look at FMI Tech Info 5673 and its related links.

Also, there is an entire tech brief on External Server authentication on the FMI web site at:

http://www.filemaker.com/support/upgrade/techbriefs.html

When a Windows user logs onto the network he/she is authenticated by either Active Direvtory or Open Directory. In your instance it seems to be Open Directory. When the user attempts to access FileMaker Server with database visibility enabled on the server, the user is challenged for credentials before being able to connect to the server to see the hosted files. FMS queries Open Directory for this authentication. If the user is valid and has a matching group from OD in the file, the user is admitted to the database.

A Windows user authenticated by OD may have his group name returned in the syntax of domainnamegroupname. Thus, instead of--for example--salesmanagers, the group name comes back domainnamesalesmanagers. The name in the file must match this.

Macintosh users cannot take advantage of Single Source Sign On or SSO. SSO is a Windows Server and Windows client process only. Macintosh users mimic SSO by using the KeyChain.

I hope this helps your understanding of this. These cross platform authentications are very difficult to diagnose long distance.

Steven

[skrying]

Thanks for the help.

I've read both of those many times, still can't find any answers.

--------------------

In the hopes that this might clarify some things:

Suppose we have two users 'mark' and 'fred', both in the OD server group 'dataentry'.

The group 'dataentry' is added to some of the files on the FM server; the files are all up and running, and the 'dataentry' group is enabled in said files and set to authenticate externally.

If either fred or mark tries to log in from a Mac client, things work as they should.

If fred tries to log in from his own Windows computer, where the local account name is fred, he gets locked out by the process in the first post. Same goes for mark. However, if mark tries to log in from fred's Windows computer or vice versa, both can log in successfully.

[skrying]

Re-reading your post, I'm wondering if the Windows clients are attempting single-sign-on? And if so... why? (since SSO is only available using Windows Active Directory, and we're using Macintosh Open Directory).

[Steven H. Blackwell]

If fred tries to log in from his own Windows computer, where the local account name is fred, he gets locked out by the process in the first post.

Probably becasue there is no matching Group in the files for him. OD is probably returning a group called domainnamedataentry. Is there a group like that? Or is teh group in the file called dataentry? Duplicate the dataentry group and rename it domainnamedataentry where domainname is name of the OD domain. See if that works.

Steven

[skrying]

But the OD server is not returning a group at all, it just says password incorrect. (but who knows which password this is because it all happens before the user ever types in a password...)

Anyway, according to http://www.filemaker.com/downloads/pdf/techbrief_fm8_server_auth.pdf on page 56, UNC syntax just specifies where to look for the account if there is a conflict. Not sure if that is relevant.

[skrying]

How's this for a curveball:

Created a new Windows account, account name doesn't match any in the OD server. Under this account, When the user clicks on the server in FileMaker, no login credentials get sent.

So there has to be something about how all the other Windows accounts are set up that are doing this.

[skrying]

Turns out this was actually just the server not logging the attempt. The credentials still get sent.

So the problem appears to be with the Windows FileMaker client.

[skrying]

Ok, last update: FileMaker support says: yes... we've heard about similar issues... hmm... don't use that feature.

[Wim Decorte]

That's what it sounds like to me, the windows clients are attempting SSO. But that shouldn't happen when the Windows users are logged into their machines with a local account.

Are you 100% that the machines were not at some point or even still are configured to be member of a domain?

Do you have the OD also set up to be a Windows PDC?

[skrying]

For the moment we've changed the FM Server to show all files instead of just the files they have access for. Now instead of the credentials being sent 10+ times, they're sent once. We've also changed the OD Server to lock out accounts after 4 failed attempts in a row instead of 3.

For the moment this works fine. We'll be upgrading to 8.5 soon, and I'll post back how things go at that point.