Troubleshooting AD Ext. Auth.

[Horatio77]

I'm trying to implement external authentication via Active Directory for our Macs and PCs with 'Single Sign On' for our Windows clients. I've tried to closely follow the FM8_server_auth tech brief but domain accounts aren't yet working. I've tried to authenticate from a Mac not on the domain, but using domain credentials when prompted, and from a PC logged into the domain, both fail. All machines are on a local network. Some clients are on FMP8, though my test machines are using FMP10.

Our Server:

Windows Server 2003 sp2

FileMaker Server 9.0.3.326

Hosting FM database: 'Mydatabase'

Client Authentication: FileMaker and external accounts

File Display Filter: List all databases

This machine is also the domain controller of: MYCOMPANY.LOCAL

'Mydatabase' Accounts:

Account Group Name: FMGraphics

Active: Yes

Type: External Authentication

Privilige Set: [Full Access] (Access via Filemaker Network (fmapp))

Client Example1:

Windows XP sp3

Filemaker Pro 10.0v1

Domain: MYCOMPANY.LOCAL

Group: MYCOMPANYFMGraphics

Client Example2:

MacOS 10.5.7

Filemaker Pro 10.0v3

Clearly SSO isn't working since I can't log in with domain credentials at all. I'm hoping it will fall into place once domain log in is functioning.

Some questions:

1) The domain group is nested in a couple of organizational units. Does that matter to the database's account group name?

2) Is there a way in Windows to get the user's group name with the exact syntax FileMaker wants? The command line tool 'whoami /groups' on the PC reports:

[Group 1] = ...

[Group 9] = "MYCOMPANYFMGraphics"

[Group 11] = ...

While the group is listed with the domain here, my understanding is that the database account only needs the 'FMGraphics' part of the name. Is that correct?

3) Is there a FMP10 version of the external authentication tech brief?

I'm frustrated because I'm not sure where even to look next. I've quadruple checked the the settings I know about, and the spelling / case of the group name, all of which I've tried to detail above.

Thanks for any tips you guys may have. This is going to be totally easy, right?

[Ted S]

Horatio,

Sounds like you're doing everything right. One question; Have you checked FILE > FILE OPTIONS and made sure the "Log in using" box is unchecked? If this is checked it *may* be bypassing the external authentication.

I'm running SSO on an all Windows network and SSO has always worked fine for me. BTW, I'm in Mpls. too.

[Horatio77]

snip...

Have you checked FILE > FILE OPTIONS and made sure the "Log in using" box is unchecked? If this is checked it *may* be bypassing the external authentication.

It is unchecked. I'm prompted for credentials every time I open the database on both platforms. This is intentional on the Mac side, having denied FileMaker Keychain access for testing purposes. On the PC side, well, part of why I'm doing this is to get SSO working. In either case, I can only log into accounts specified directly in the database.

None of the databases local accounts have names that overlap with domain users.

I've noticed on the Windows machine it auto enters my domain long name in the 'user' log in field. I'm not sure it means anything, as it works with neither the short nor long anyway.

[Steven H. Blackwell]

Your Groups don't match:

MYCOMPANYFMGraphics

is not the same as FMGraphics.

Try making a Group in the file named MYCOMPANYFMGraphics and see what happens.

Meanwhile, a few other items:

Don't run FMS on the Domain Controller.

Be sure that the FMS machine is bound to the domain.

Don't authenticate a [Full Access] account externally. If someone got a copy of your files, they can possibly spoof your domain.

There is Server 9 version of the External Authentication Tech Brief.

HTH

Steven

[Vaughan]

To help with de-bugging, it's useful to know exactly which account and privilege set the user is logged-in with. This can be done with a simple script with a custom dialog.

Remember that the FMP file can possibly be opened with an account from one of three places, and in this order:

1) an account inside the file itself

2) an account on the FM server machine

3) a group in the AD/OD.

[Horatio77]

Your Groups don't match:

MYCOMPANYFMGraphics

is not the same as FMGraphics.

Try making a Group in the file named MYCOMPANYFMGraphics and see what happens.

That was one of the first things I tried, along with including the 'Organizational Units' (folders?) from Active Directory. I didn't have any success, but may have switched the slashes or similar.

MYCOMPANY in this case is our domain, which the 'whoami /groups' command includes to distinguish them from the client machine's local groups. Looking closely at the FM8 tech brief, the domain name appears to be excluded when calling an external group for authentication, which is why I went back to simply 'FMGraphcs' for my original example.

Meanwhile, a few other items:

1) Don't run FMS on the Domain Controller.

2) Be sure that the FMS machine is bound to the domain.

3) Don't authenticate a [Full Access] account externally. If someone got a copy of your files, they can possibly spoof your domain.

4) There is Server 9 version of the External Authentication Tech Brief.

HTH

Steven

1) I am not involved with the FMS and DC deployment, and changing them is beyond the scope of my position. Perhaps after I show my boss how awesome I am by getting External Auth to work. Is there any reason this set up would interfere with authentication?

2) A quick check of the domain on the FMS host system via 'My Computer' properties reports a 'MYCOMPANY.LOCAL' domain. Being that FMS and the DC are currently on the same machine, is it possible for FMS not to be properly bound to the domain?

3) Thanks for the reminder about [Full Access]. I have been using that to make sure I don't have something unexpected in my privilege set to gum up the works. I'll make sure to apply a restricted privileged set once I get any domain account to work.

4) After a little digging I see that a Server 9 Tech Brief is available through TechNet. I don't have a subscription so I guess I'm out of luck there.

Thanks for the pointers and reminders. It seems like I'm digging around the right places, I just need to keep at it.

[Horatio77]

To help with de-bugging, it's useful to know exactly which account and privilege set the user is logged-in with. This can be done with a simple script with a custom dialog.

Remember that the FMP file can possibly be opened with an account from one of three places, and in this order:

1) an account inside the file itself

2) an account on the FM server machine

3) a group in the AD/OD.

An excellent tip indeed. In fact I already have a script of this very nature as per your suggestion to others on this forum.

Unfortunately, I've yet to get domain credentials to log me into anything in FM, so the script hasn't had a chance to show its utility on this problem (it's gotten plenty of use elsewhere.)

[Horatio77]

I've taken a step forward, having successfully logged into my database using Active Directory credentials and SSO. Unfortunately it was by changing the FileMaker account group to 'Domain Users' (definitely NOT with [Full Access]). So at least I now know my problem lies with our AD groups rather than syntax or binding problems.

I'll be turning to our IT contractor for some help now, but perhaps the following picture will show something amiss to those here familiar with AD.

This highlights the group I want to authenticate.

The 'Domain Users' group I logged in with resides in the 'Users' container at the very bottom of the console tree. I'm not sure which factors make one group work and not the other. I note that the 'Users' container is the same place Wim Decorte created his groups in the tech brief rather than something like the 'My Company/Groups' organizational units we have set up, perhaps that matters?.

Also, the 'Domain Users' group is itself a part of the builtin 'Users' group and the 'CERTSVC_DCOM_ACCESS' group. My 'FMGraphics' group does not belong to any other groups. This is well above my head.

I'll update when I finally crack this.

[Wim Decorte]

To see if nesting the group in the OUs plays a role: test with a group name that is not nested but sits in the regular users & groups section of the AD.

Since the FMS machine is also the DC there can't be an issue of the FMS machine not being bound to the domain. But for good measure: do try to log into the FMS machine itself with a non-admin AD account that belongs to your group.

What do the FMS event logs and the application and security logs on the FMS/DC machine say? You should see entries for the times you've tried to log in.

Are the workstations properly joined to the domain? Can you log into them with a domain account (making sure that the same account does not exist on the workstation itself). Do the workstations show up properly in the "computers" section of the AD?

On the workstation, after logging in with a domain account that belongs to your target group, go to the command prompt and type 'gpresult'. At the end of that output you'll see a full list of groups that user belongs to. Is your group listed? Can you paste that result here?

[Horatio77]

Thanks for the tips Wim. The event logs on the DC were indeed showing errors.

I've been talking with the technician who deployed our Domain and FileMaker server system. Turns out we have two machines serving as DCs, and the groups weren't propagating between them properly. When the groups were manually created on both DCs then FileMaker's SSO worked as it should.

Now we just have to get our DCs in order, but that's for another forum (and the technician we contracted to do it in the first place).

Thanks for the help everybody.