Where to store lots of images?

[Peter1]

I have a small publishing business. All adverts are currently emailed to customers for approval (as low res JPEGs). I want to be able to store these on a separate computer (I have some old iMacs which could be used for this). Where do I start to set this up?

If I create a separate database file to store these I would accumulate about 5000 images per annum - equalling about 60MB. I would want to keep about 5 years back-referenced, so the file would get to about 300MB max. What is the best option? How much slower will it be for users to access if not embedded? Users would ideally also view results in a portal (smaller image size) - is it worth creating a second smaller thumbnail for this purpose? On a Mac does 72dpi still apply or is 96dpi better quality (at 100% scale)

Thanks - Peter

[Barbecue]

If you mean you want to store the images on a separate drive from the database itself, that will definitely slow things down, but how much will depend on several factors, including the speed of the computers, the quality of your network, and how often the images themselves must be retrieved.

In my experience using Windows, rendering the image to the screen is a bigger bottleneck than the actual transfer of the image data across the network, so the biggest thing I do to improve performance in a database with lots of images is to keep the images off the main layouts used for browsing, and only display them "on demand" via a button. I've heard OS X is a lot better in this regard, but I don't have any experience with it.

Thumbnails might help, but it sounds like the images you're storing are pretty small already, so I'd be more concerned about keeping things simple.

Rather than worrying about DPI for thumbnails, I'd suggest picking an arbitrary size in pixels or an arbitrary percentage.

[Peter1]

Thanks for your response.

The images will only be available on demand. They will be accessed by users about 5 times per day. One of my customer layouts is a portal of advert thumbnails (very small - 10k each) and linking to the advert (40k to 250k).

For many reasons it is easier for me just to reference the file from FileMaker, as the images are accessed for other non-FileMaker reasons. However, if there is a significant slowdown then I do want to store them in FileMaker. I would ideally like to test both ways and do some time tests, but do not know what is required to access the files. I read somewhere that it is a problem if the FileMaker server also stores the files and that it is better to store them on a dedicated computer - how do i set this up?

Thanks - Peter

PS: Congratulations on recently becoming a grandfather. In 10 weeks time I am becoming a father. Very exciting.

[Fenton]

I would sort of agree with Barbecue, that your images are small enough to store directly in the database. However, your use of the larger images (40-250k ones) is to email them, which means handled as files. So perhaps it would be better to store them on another machine, with File Sharing on. Then anyone with access can mount that volume, and attach files to email messages.

Of course you could do the same with files embedded in the FileMaker database, as files; embedded with Insert File. You would use Export Field Contents to get the file out. If it was exported out (moved) to the user's "temporary items" folder, then it could be attached to an email and sent, and the extra file would be automatically deleted when you restarted (I think that's when it gets dumped).

Since they're low resolution jpegs, you could also export an embedded jpeg and save as a file, using AppleScript.

That all sounds complicated, but it could be handled with AppleScript, so that a user wouldn't have to even know what was happening behind the scenes.

Open Script Editor. Type:

path to temporary items

I've messed about with this a little bit. But not really enough to give conclusive advice on which would be better or faster. Either method would work. Each has its pros and cons. One allows you more freedom to access the images as plain old files; but requires that you keep the paths synchronized with your FileMaker database. The other gives you less access to the files as plain old files and folders, but keeps them synchronized.

The first needs a good mechanism to keep the files in sync, the 2nd needs a good method to get the files in and out. Another consideration is how much editing of the files you're going to do. Either method of storage can handle that, but it would be different.

With 7 there are a few options. It's kind of hard to say which is best; and, as I said, I've not done any in mass production; but I know there's more than one way that works. Perhaps you should experiment a little with each.

One option that does NOT work is to store the referenced pictures on the computer with FileMaker Server, not unless you turn File Sharing on for that machine; which is not recommended.

[Peter1]

Hi Fenton, thanks for such a detailed response.

I am going to mess around and see what works best. I am going to do some "time-scroll" tests on 1000 images (100k) in list view referenced to a separate computer and the same on 1000 images in list view embedded in containers. It will be interesting to see how long each takes to scroll from top to bottom of page. Thanks again for the help.

Peter

[Fenton]

My response may have seemed detailed, but I'm afraid it was just an overview, and far from conclusive. I'm fairly sure the embedded ones will scroll faster. One thing I didn't mention in my earlier post was that, unless the source files are going to change drastically, I would always embed the Thumbnails, like 128x128; to use in List views.

If you're going to go with referenced files, you should also consider using a calculation to show the image. We've covered that several times here in this forum (but not conclusively :-).

That's the other dimension of storing images; how you actually interact with them; what kind of structure the folder hierarchy of images has, and whether the database can help create and maintain it. If you're on a Mac, then AppleScript can do quite a bit of this. Or whether the database must just follow the existing structure outside its control.

Because, as you've undoubtedly found, FileMaker can either export to, or import from, a known fixed folder; or ask you at the time. It doesn't support developer scripting access to its File References (which would be so great, the ones for import and export especially, which are only stored in the script step itself, not in file's main File References).

[Peter1]

I have read several of your discussions on this topic and will continue to do so for a few days. I know with the help above and elsewhere in the Forum category I will work out what's best for my needs. Watch out for future posts.

Thanks for your time spent on this. Peter

[Barbecue]

My own informal benchmarking of stored versus referenced images in a Windows FM7 database indicates that browsing stored images is roughly twice as fast as browsing referenced ones when the database is local. When the database is on an FM7 Server, the improvement is less noticeable.

I created a loop script that simulated navigating a set of records with an arbitrary collection of 58 images ranging in size from 10k to 1.6 megs, running 2000 iterations.

I ran the process four times, once as a local database with images referenced instead of stored, once as local with images stored, once from the FM server with images referenced, and once from the server with images stored. All image references were to files stored on a separate file server.

Local DB

images referenced: 169 seconds

images stored: 81 seconds

-------------------------------

stored images are ~100% faster

Server DB

Images referenced: 161 seconds

Images stored: 107 seconds

-------------------------------

stored images are ~50% faster.

Observations:

The first-time run through the images took dramatically longer than subsequent repetitions, whether the images were embedded or referenced. This is not surprising, since both FileMaker and the network server use cache to speed up repeated reads.

Conclusions:

For a FM Server based solution, the advantage of storing versus referencing images is partly offset by the time needed to push the actual binary image data to the client across the network. The difference is measurable, but not that noticeable to a user working with the file.

In cases of large databases where any arbitrary image may be browsed, performance will probably be significantly slower than these benchmark figures indicate.

[stanley]

Another option would to be to do both. The stored files are faster, for a number of reasons, as stated above by Fenton & Barbecue. However, if the files are to be used in other apps, you would want them to reside on a file-sharing volume, entirely separate from FileMaker. You can point to the "real" file via "Open URL" if you want (there are a number of other ways to do this, too.) The advantage of doing both is that you can treat the stored images as backups, in case someone mistakenly alters or deletes one of the source files on the shared volume.

-Stanley