Jump to content
Sign in to follow this  
Ocean West

Windows Server Bash / FTP - or Automated FTP sync option?

Recommended Posts

I am reworking my process of using FMP server side script.

  1. Run Server Scheduled Script every 45 minutes
  2. Log in to FTP get directory listing (plugin / curl)
  3. Create Record for each file in listing
  4. Fetch file via (plugin / curl) to text field
  5. Write file to server's Document folder
  6. Move file on FTP to sub directory (compete)
  7. Import files from document folder in to table

It works but think kludgy and FMS probably requires more overhead to deal with the FTP plugin.

Is there a way to do the following

  1. Run a server sequence script every 45 minutes
  2. Run a bash script on server login and fetch new files from ftp (need help with this)
  3. Save files to FMS Documents/incoming
  4. Run script that gets incoming files
  5. Run script that imports files
  6. Run script that moves files to complete directory
  7. Run bash script with file listing as parameter (need help with this)
  8. Bash script logs in to FTP moves files to complete folder on FTP OR DELETES them (need help with this)

I am hoping this would be much more efficient for FMP to only have to do file imports vs having to also FTP. 

Thanks SD

Share this post


Link to post
Share on other sites

Why not use the "insert from URL" with the FTP cURL options?  No plugins needed.

Share this post


Link to post
Share on other sites

The problem  with using insert from url I would still have to capture the file as text or binary then write it to the os just to perform an import. 

And then at some point move or delete the remote files  

My thought was in FMP only have to get directory contents and then import and move. 

 

I also couldn’t find examples of the curl syntax to move or remove files 

Share this post


Link to post
Share on other sites

There's going to be a fair number of moving parts, whichever way you do it, by sticking with "insert from URL" you keep the moving parts in FM's space with fewer outside dependencies.  And better error trapping and handling.

 

Share this post


Link to post
Share on other sites

So I was able to make connection get file lists and loop thru and download files, with curl.

Been googling all morning can't find the correct settings for the following...

  • seems like every call i have to pass user name & password - (plugins seem to have a starting / ending routine)
  • is there a disconnect command
  • can't figure out how to move / rename / delete remote file
  • with curl there is no direct way to output file to desktop still need a field or variable to hold the raw data.

 

Share this post


Link to post
Share on other sites
Posted (edited)

Depending where the ftp server is located maybe forgo ftp and mount the remote system to a folder (using SMB or NFS or whatever is available, and more secure) and use BaseElements to manipulate the filesystem? You could permanently mount it or mount on demand in the script (also using BE).

Much simpler than messing with ftp. That's so last century.

Edited by OlgerDiekstra

Share this post


Link to post
Share on other sites

Or even write your own little mini web service (microservice) to deploy on that remote machine.  We do that a lot in the case where files get produced from one system and need to be imported into FMS.

Real-world example:

- "other system" drops thousands of XML files to a server through an MQ messaging system

- small custom Windows service watches the folder and every time a file gets added, parses it and adds it to a small Sqlite db (all very lightweight but it can handle thousands of files being dropped in the watched folder at once).  This service could have just turned around and push the data to FMS through the Data API, XML API, PHP API or ODBC/JDBC but we wanted the freedom to de-couple both mechanisms (receiving the files, and adding the data to FMS), so we added:

- small custom web service (ASP.NET in this case but could be any technology) basically provides a GET endpoint that will read from the Sqlite database and send all the data in JSON format to the caller and then delete it from the Sqlite db (so that there is no persistent storage in  sqlite)

- over on the FM side there is a schedule that calls that web service every 5 minutes.

This whole setup was built to replace a standard FM import-and-process routine that was too slow to keep up.  It's blazingly fast.

Because we chose to add that Sqlite db in the middle, we are free to shut down FMS for as long as we need to, without interrupting the incoming flow of data from the other system.

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Sign in to follow this  

  • Similar Content

    • By William
      Why would I be getting this on install?  MacOS Catalina, latest MacBook Pro, just recently installed Python 3, Django, PIP, Eclipse which had me install the latest JDK from Oracle.
      Thanks!

    • By Joost Miltenburg
      Hi All,
      I added security to limit the DataAPI user from accessing certain records, with the limited option. I tested this while running these files on my HD no server. Worked like a charm. After moving the files to FMServer the DataAPI account can access all record in the table all of a sudden.
      Any thoughts as to why this is happening?
      Kind regards,
      Joost
    • By William
      I'm having trouble with FMPA 18.0.3.317 creating a runtime.  Even with the simplest file, I create it and when it runs it immediately crashes with the following in the crash report:
      Exception Type:        EXC_CRASH (Code Signature Invalid)
      Exception Codes:       0x0000000000000000, 0x0000000000000000
      Exception Note:        EXC_CORPSE_NOTIFY
      Termination Reason:    Namespace CODESIGNING, Code 0x1
      I did the exact same thing with FMPA 17.0.6.600 and it worked fine.
      Any suggestions as to what's going on here and how I might fix it?
      Thanks!
    • By hollyheadhunter
      If a user accesses a file via WebDirect on a mobile device, what is the easiest way to let them take a photo and capture it in the system? 
      Do they have to take a photo outside of the file and then upload it, or there a faster/better way to take the photo and capture it in fewer steps?
    • By Madapple
      having trouble with calculating related fields
      scenario: inventory bin consolidation app
      Item 3AT17 is in 5 locations - 2 locations have just 1 unit, other 2 have 36, 65, 94 units. we want to move the two single units to the 36 unit bin. (see below)
       
      the locations/item info is in a portal. The move to list  from the related locations field - showing all locations with the item. 
      How do I calculate the new total for the "move to"  field? 
       
      Layout is pulling from Item Master (IM_) with related tables Count (CO_) and location (LO_)
      Relationships:
      IM_ITEM=CO_ITEM
      LO_Location = CO_Location & LO_ITEM = CO_ITEM
       
      Field Names:
      Location:  CO_LOC
      item : CO_ITEM
      QTY: CO_QTY
      move:CO_RELOYN
      Move QTY: CO_RELOQTY
      Move To: CO_RELOTO
       
      For the above example moving 1unit from 2 bins - into third bin need a cell for the new qty in that bin. 
      so in this example new qty would be 38.
      I am sure I am missing something that would help with figuring this out... 
       
       
  • Who Viewed the Topic

    10 members have viewed this topic:
    org  Robin  Karsten Wolf  Lape  docasar  Aussie John  bcooney  doughemi  Steve Martino  Lee Smith 
×
×
  • Create New...

Important Information

By using this site, you agree to our Terms of Use.