Jump to content

Matching similar records in diff. tables


MikeAir
 Share

This topic is 2933 days old. Please don't post here. Open a new topic instead.

Recommended Posts

Hi,

 

I am currently writing my bachelor thesis in economics, where I am conducting an analysis of 105.000 small text messages, which i have chosen to organize in a filemaker database.

 

The observations has serveral fields; ID, Name, Group, Message, URL.

 

I currently have all the data stored and sorted in my filemaker database, but the next step is to conduct a sentiment analysis using the software called rapidminer. This software only need the 'Message' field, and another column named 'Sentiment' where positive messages is mapped with 1, and negative with 0. 

The point with using rapidminer for this analysis is, that I only need to map a few thousand messages, and thereafter rapidminer will do the rest of the work based on the pattern between words and sentiment.

 

Here is the problem:

 

Rapidminer only outputs the Message & Sentiment fields afterwards. This means I will have 105.000 additional observations with a sentiment mapping, but with no 'ID' 'Group' or 'Name'.

 

The point is, that i need the data outcome from rapidminer back in filemaker, organized in the right way with both the same ID and Name as before i exported them to rapidminer for the analysis.

 

> Some months ago a kind person on this forum(eos i think his username was) helped me build a script that could recognize if two fields in two different tables had the same numerical value, and thereby show them in a portal, making it possible to give Obs1, the ID# of Obs2, as a foreign key. This was brilliant, and worked fully with numbers.

 

However, does any of you think that this will be possible with text fields as well to help me solve my problem?

 

The reason i need to join the data after the sentiment analysis, is that i need to do some further statistical analysis on the other variables in some other software.

 

Kind regards from DK

Mike

Link to comment
Share on other sites

Rapidminer only outputs the Message & Sentiment fields afterwards. This means I will have 105.000 additional observations with a sentiment mapping, but with no 'ID' 'Group' or 'Name'.

 

There are several ways you could handle this. If RapidMiner returns the records in the same order as input, then the simplest (and fastest!) method would be to export the records in a known order (say unsorted). Then import the result back into the same table (mapping only the Sentiment field to a target field) and select "Update existing records" as the import method - see:

http://www.filemaker.com/help/13/fmp/en/html/import_export.17.11.html#1027964

 

Make sure you have a backup while you experiment with this.

Link to comment
Share on other sites

But unfortunaly that is not the case..

 

Well, then you need to move up to the next level of complexity and import the records using the "Update matching records" method - matching Message to Message and importing the Sentiment. Still very simple to implement, only slower to execute.

Link to comment
Share on other sites

Well, then you need to move up to the next level of complexity and import the records using the "Update matching records" method - matching Message to Message and importing the Sentiment. Still very simple to implement, only slower to execute.

 

I just made a test out from a set of dummy-observations containing 4 fields and 10 rows, imported them to a new filemaker DB, then made a copy of the observation set, added some new data, manipulated the order of the records, and imported them using your instructions^.

It worked!! Thank you.

 

I love this forum.

Link to comment
Share on other sites

This topic is 2933 days old. Please don't post here. Open a new topic instead.

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share

×
×
  • Create New...

Important Information

By using this site, you agree to our Terms of Use.