When i use Excel i can place VBA code into the Before_Workbook Close event, to establish if there are Other workbooks open.

If there is the active workbook is closed.

If there are no other workbooks opened, then Excel is closed off completely(exit application).

Is there a way to do this in Filemaker? To either Close the file or Exit the Application, depending on whether another file is opened?

Look in the File Options (in the File menu) there is the options to set scripts to run at file open and file close.

I understand the File options when a file opens/closes, i do not understand how the calculation is applied to find if another file is opened though.

Does it need to refer to the file names, or is there a calculation function that refers to files if they are open?

You can return a list of open files with DatabaseNames from the Design functions suite.