No authentication: FM Server on Windows, DC = OpenDirectory

[ma_at_fmf]

Dear Forum,

currently we run FileMaker Server 8 on Windows 2000 Server which authenticates against ActiveDirectory. It works fine.

Now we want to change authentication against OpenDirectory. With the File Maker Windows Server I left the AD-domain and joined the OD-domain. According to the "Server Authentication" guide, the only relevant configuration option in File Maker Admin is, where you have to give the external server's IP or name. I changed it from the AD-DCs IP to the OD-DCs IP.

By OD-Workgroup-Manager I added all groups to the OD, also existing in AD. I assured that the groups are also defined on the databases (of course, it wouldn't have worked against AD either, if they weren't).

Now, authentication still doesn't work when I try to connect from a client's FileMaker Pro to the database.

It's just an assumption, but I guess that the FileMaker Windows Server cannot find the OD-groups. When I run "net user /domain" from the FileMaker Windows Server's cmd, it shows all OD users. Running "net group /domain" doesn't find any OD groups.

Any ideas?

Thanks in advance!

[Steven H. Blackwell]

An interesting puzzle. And I have alerted the world's leading expert to the posting of this thread.

A few possible items:

1. Windows Seerver 2000 running SP 4?

2. OD uses the short name. Is the short name in the group name in the file?

3. Are you running the Server 8.0v4 updater?

Steven

[Wim Decorte]

In addition to Steven's questions: can you successfully log in on the FMS machine with an OD account? If you can, what group does it say you belong to?

Have you disabled all the existing AD accounts/groups to make sure they are not being used?

It's not totally clear to me what you mean by the FMS configuration and the name or IP address you changed. Where exactly did you do that? What's the settings name?

[ma_at_fmf]

Thanks for your answers.

@OAM

1. Yes, the OS is with SP4 and full updated.

2. Which file do you mean, according the short name? Where can I define or check it?

3. Yes it is File Maker Server 8.0v4.

@WD

1. Yes I can login to the server with an OD account. It says nothing about group membership. Where can I check it?

2. No I didn't. The production environment is still in use until the migration. The FM server is setup on a test-machine, being member of the OD-domain. IMHO the problem has nothing to do with the FM Server being confused between AD and OD groups. It even doesn't know anything about DNS and WINS of the AD-based network.

3. In File Maker Server Admin, there's the Tab "Verzeichnisdienst" (=Directory Service), when you rightclick on and choose Properties. Here you can give the name or the IP of the external authentication server FileMaker Server should use. Or am I wrong and this prompt also has nothing to do with authentication? But where can I define, against which external authentication server FileMaker Server shall authenticate users, if this has nothing to do with authentication? Or does FileMaker Server know simply because of its own domain membership where to authenticate users against?

@all

Another indicator that my assumption might be right (that the FileMaker Server doesn't find the OD groups) is, that the OD account I logged in with on the FileMaker Server, belongs to the users with domain administrator privileges in the OD domain. While the FileMaker Windows Server added the DomainAdmins' group (this group doesn't exist in OD, so it must be a Windows-thing, a standard group which is added to the local Admins group when you join a domain) to the local Admins group on the Server, the user didn't have local admin privileges until I added his user-account manually to the local Admins' group.

[Wim Decorte]

3. In File Maker Server Admin, there's the Tab "Verzeichnisdienst" (=Directory Service), when you rightclick on and choose Properties. Here you can give the name or the IP of the external authentication server FileMaker Server should use. Or am I wrong and this prompt also has nothing to do with authentication? But where can I define, against which external authentication server FileMaker Server shall authenticate users, if this has nothing to do with authentication? Or does FileMaker Server know simply because of its own domain membership where to authenticate users against?

Hi Betty,

That's a misconception. What you put under "directory services" in the FMS configuration has nothing to do with authenticating users. That section is merely for information purposes, like putting the FMS info (IP address, who's responisble,...) in the phonebook.

It is the FMS OS configuration that decides where FMS will look for authentication. If the machine belongs to a domain, it will contact the domain controller. If it is not part of a domain, it will look in its local accounts.

I suspect something is wrong with the domain membership of the FMS machine. When you look in the AD, do you still see the machine listed as a member server or member computer?

[ma_at_fmf]

@WD

Okay, I removed all heritages of the AD-domain. AD, DNS on AD and WINS know nothing about the FileMaker Windows Server and the FileMaker Windows Server knows nothing about the AD-domain, its DNS- or WINS-servers. The problem persists.

Playing with the domain-membership of the client which runs FileMaker Pro (this shouldn't make any difference, because apart from single sign on scenario, you should be prompted for a privileged user account, don't you?) made me even more confused: If the client belongs to the OD-doamin, the OD-groups are found and authentication works. I also played with the OD-group membership of the OD-user account and noticed that in fact authentication works for user accounts being member of defined groups on database level and fails for user accounts not being member of it.

Unfortunately, it's not an option for us, migrating all workstation pcs to the OD, because the AD branch hosts the Company's administration users and computers while the OD branch hosts the education and production sections. Independent of the workstations' domain membership, access has to work for all users (all users own both an OD- and an AD-account. If we could migrate authentication of FileMaker to the OD, we could decrease user administration work significantly, because the users of the education and production sectors wouldn't need to have an account in the AD. They only have it to be able to use the FileMaker databases.).