Importing Data¶

How Imports Work¶

To import data, first create a CSV or TSV file, following specific rules for creating an import file. Your file can include both new and existing users. Then create or select an import page and upload your file. Read the how-to for details.

Imports create actions - most imports will create actions, even if you're just updating information for existing users. The only exception is custom userfield updates with Skip Action Creation enabled.

Each time you upload a file, a row is added to core_action for each row in your file. The action is assigned to the user_id corresponding to the email (or akid or user_id or texting_address) in each row.

Most imports are processed through the Import page type. Events are the only exception -- you can upload individual events to Event Create pages and upload individual event signups to Event Signup pages.

Import actions are much like other actions. For example, you can target a mailing to users who took action on an import page. Import actions count as actions in most built-in reports.

You can specify source and other action related data. For each new email in your file, a new user is created and the user_created field in core_action is set to 1. See the New User, imported report for a list of new users imported in the last week.

As with other actions, new information submitted overwrites existing data for fields in core_user, and core_phone, but only if there is new data. You cannot overwrite existing data with an empty field. For example, if you include a zip column, the value in your file for a specific user will replace the user's current zip unless that column is empty. An exception to this is core_userfield, where a blank value will cause any existing values for the given field to be deleted.

Finally, by default each action subscribes the user to the list you select when creating the page, unless they're already on that list. Unlike with other page types, you can change this for import pages using the Subscriptions setting. It's important to set this intentionally before you upload your file to avoid subscribing users who have asked not to be emailed.

Product Orders¶

Product orders are a type of donation. The rules above for donations apply. To import sales of products, you must first create the product from the Pages Tab.

The following fields are required:

• donation_product_ID - (Required) Where ID is replace by the ID of the product. For example, if you were importing sales of t-shirts and the ID of the t-shirt product was 75, the column name would be donation_product_75.

  Values for this column can be either a quantity number (ex. "10") or a pair of numbers giving quantity and total paid separated by a pipe (ex. "10|50.00").

  If the price for your product in ActionKit is accurate then you can use just a quantity and the amount paid will be calculated for you. However, if the price doesn't include everything (i.e., if the total also includes shipping charges), then use the compound format above and include the actual total amount paid by the user.

• donation_amount - (Required) The value must be at least as much as the total of all product columns.

  The value can be more; that's how ActionKit stores tips on product pages. For example, if you have product orders for $5.00 and $10.00, donation_amount must be at least $15.00. You must include this column even if it equals the total for products.

Candidate Contributions¶

Just as with products, you must first add the candidate from the Pages Tab. Then your file must include:

• donation_candidate_ID - (Required) Where ID is replaced by the ID of the candidate (e.g., donation_candidate_20 for candidate ID 20). Enter the amount of money donated to the candidate in this column (e.g., "9.99").
• donation_amount - (Required) The value must be at least as much as the total of all product columns. The value can be more; that's how ActionKit stores tips on product pages. For example, if you have product orders for $5.00 and $10.00, donation_amount must be at least $15.00. You must include this column even if it equals the total for candidates.

Recurring Donation Profiles¶

You can import recurring donation profiles. You cannot import individual payments toward a recurring commitment.

In addition to the normal donation import fields, you must include fields for:

• donation_amount
• recurring_id - the profile ID used by the payment processor.
• recurring_payment_account - If you have more than one payment processor configured you must include the name of the account which should be associated with the recurring profile. If you do not include this column or the value does not match one of your payment accounts an error is returned.

You may include:

• recurring_period - Choices are years, quarters, months, weeks.
• recurring_first_payment - By default an initial payment is created when you upload your recurring profile. If you do not want to create an initial payment at the time of your upload, put a 0 in this column.
• recurring_start_date - the date of the first payment on the profile.

User Language¶

Languages must be added from the pages tab before they can be used. Read about setting up and using languages.

English is the default language. Set a different language for any user by including a column with the lang header.

ActionKit recognizes:

• language name (e.g. "English", "Espanol", etc.),
• ISO code (e.g. "en", "es", "ru", etc.),
• ISO name (e.g. "Spanish", "Russian", etc.).

To find the 2 digit ISO code and the official name click Language and Languages from the Pages Tab, then click Edit and look at the Language code field. We suggest setting this code for all your languages to take full advantage of ActionKit's language functionality.

You can also change the default language for the page when you create it. The default applies to all users taking an import action through the page, unless you've got a different value for one or all rows in a lang column in your file.

If you don't want this import to affect the language entry for existing users, you must change language from English to ----------- when you create the page.

User Phones¶

Phone numbers are saved to the core_phone table. Users can have one phone number of any type per source. Uploaded entries have a source of admin. Your entry will overwrite any entry you made previously but not an entry from the user, which has the source user.

• phone_type - (optional) Specify a type if desired. If you don't enter a type, the default is home. Phone types are:

  TYPE_CHOICES = (('home', 'Home'),
            ('work', 'Work'),
            ('mobile', 'Mobile'),
            ('emergency', 'Emergency'),
            ('home_fax', 'Home Fax'),
            ('work_fax', 'Work Fax'))
  

• Alternatively you can include the phone type in the header. This is useful if you're only uploading numbers of one type in your file. The accepted phone type headers are: home_phone, work_phone, mobile_phone, emergency_phone, bat_phone.

• phone - User's phone number. Entry is not validated. ActionKit will save the number with no letters or punctuation to the core_phone.normalized_phone field.

Custom Fields¶

You can import custom actions fields and custom user fields. You must use the correct syntax or your data will not be saved.

• Action fields - Your column header must start action_, followed by the field name; an example is action_comment. The field name can only include letters, numbers, and _ (no spaces or other characters). You do not need to create the field first.
• User fields - Your column header must start user_, followed by the field name; an example is user_occupation. The field name can only include letters, numbers, and _ (no spaces or other characters). You must create the user field from the link on the Users tab first, or leave the auto-create user fields option checked when performing your upload.

Custom user field validation and defaults are not applied to imported values.

Tagging Users¶

You can associate tags with an import page and all the users imported through the page will also be associated with the tag(s).

Adding / Removing Users to/from User Groups¶

You can add a user to a user group, using either the group or group_id column (not both). Groups must be created in advance before adding users to them.

• group needs to match the name of a user group.
• group_id needs to match the ID of a user group.

You can remove a user from a user group, using either the group_remove or group_id_remove column (not both).

• group_remove needs to match the name of a user group.
• group_id_remove needs to match the ID of a user group.

Setting Subscription Status¶

As with other actions, import actions subscribe users to a list by default.

• Subscribe to list - By default when you create a new import page this is selected along with your default list in the List box. . Users subscribed by upload will have a change_id of 9 in their core_subscriptionhistory entry. The created_at timestamp in the core_subscription and core_subscriptionhistory tables will be set to match created_at for the action that subscribed the user.

You can change this. The choices are:

• Subscribe to list only if new to database - Users already in your database will not be subscribed; completely new users will be subscribed to the list you provide.
• Don't change email subscriptions
• Unsubscribe from list - Select the appropriate name in the List box.
• Unsubscribe from all lists

ActionKit Admins can also change the default setting for new Import pages from "Subscribe to list" to "Dont' change subscriptions". To do this, go to the CONFIG menu and edit the Uploader setting in the Pages section.

Note

You can resubscribe users who unsubscribed. You might use this if someone accidentally unsubscribed and then asked to be resubscribed. Be sure you only resubscribe those you intend. If you end up mailing a user who asked to be removed from your list this can quickly hurt your email reputation and your deliverability.

You'll receive an error message and the user will not be subscribed if you try to resubscribe users who bounced or marked your email as spam.

If you want to subscribe any new users, but leave other users as is, add a created_at column with a prior date. ActionKit won't resubscribe users who unsubscribed between the created_at date and the date of the import (which is saved in the updated_at column). For example, if you're importing a list of users who took action at an offline event 3 days ago, you'd use the date of the event as created_at.

If there are new email address in your upload file and you do not select the box to subscribe them to a list, the new users will have a subscription status of Never and no row is added to core_subscriptionhistory.

Texting Subscriptions¶

If you use ActionKit's Text Messages system, you can also use imports to subscribe or unsubscribe texting users.

• Don't change texting subscriptions - Add the new user (if necessary), but do not subscribe or unsubscribe them from texting
• Subscribe to texting list - Add and subscribe any texting_address from found in the import file to receive SMS and MMS broadcasts from ActionKit. Note that, unlike email, there are no separate "lists" for text messages.)

Note

A texting user may be imported without supplying an email address. In this case, a fake (placeholder) email address will be generated for the texting only user. This is necessary for the ActionKit ecosystem which requires an email address to primarily identify all users.

Alternatively, a new texting user can be automatically matched to an existing email user. (See Match Texting Users below.)

Match Texting Users¶

Due to the different way that texting users are acquired (texting users might be added by texting_address only -- no email address), it is possible that someone who is already subscribed to one of more email lists is added by texting_address only. You can attempt to automatically match any existing email users with imported texting-only users, by comparing the texting_address to any existing phone numbers associated with the user's record.

This matching function is enabled by checking the box next to Match Texting Users in the Import Options section, when editing an import page, and selecting the various match settings.

Match Texting Users settings

Following are the options for finding user matches between email users and imported texting users:

1 Maximum Phone Age -

Users may change their phne numbers over time. The more recently a phone number was added, the more likely that it is the correct phone number for that user. This option allows you to set a maximum "age" (in days) of a matching phone number. After this many days from when the phone number was added, the match will be regarded as too unreliable to be allowed. It is recommented that you not exceed 365 days (one year).

2 Name Match -

Available options are None, Exact, or Fuzzy. None removed the requirement for matching name, at all. Exact requires that the first and last name exactly match in terms of spelling. (Match is case-insensitive.)

"Fuzzy" is the most complicated option. A name match is considered successful at a "fuzzy" level in the following circumstance:

• Spelling of first and/or last name does not match, BUT the names are similar-sounding. (This option uses the "SOUNDEX" system. See https://en.wikipedia.org/wiki/Soundex )
• Either the first or last name does not match (or sound similar), but the other name is an exact match.

Note

You must provide columns first_name and last_name in your import file to utilize any form of name matching

3 Match Location -

If "Match Location" is selected, either the zip (postal) code must match, OR city and state must match.

Note

You must provide columns city, state, and zip in your import file to utilize location matching

Configure the Import¶

1 Create your file for upload using the required formatting and headers. See the relevant section for header names for importing donations, user data, etc.

2 Click Import Users on the Users tab.

If you're making a new import page, enter the title and shortname.

Alternatively, you can use an existing import page. Follow the Click here link in the description text underneath the Import basics title and select the appropriate import page from the listing.

Generally, you'll want to create a new page if you want to change the settings and particularly the tags. You'll want to reuse a page if you want the new actions to show up on an existing page (for example, for targeting a mailing to actiontakers).

3 Confirm the settings described below.

Tags: Select if you want to tag users.

Subscriptions: Consider whether your file includes new and existing users and when they took action if that's applicable. Read about the options for setting subscription status.

Mailing List: This setting will display if you've chosen an option that subscribes users above.

Default Action source: Enter a source in the Default action source box to set a page-level default for this page. This will apply to all actions uploaded through this page unless you include a source column as noted under importing actions.

Include in Reports of Member Actions: Check this if you want actions on this page to be counted as member actions in reports.

This allows you to distinguish between actions generated from users interacting with your pages, versus administrative-type actions such as updating custom user fields.

By default, Import pages are not checked.

Language: Toggle open the Multilingual Options section to select the default language setting for this page. If only English is available, click the green + plus sign to add a new language.

You can set a default for this page, set language by row, or leave language as is for existing users as described under user language.

Multilingual campaign: Toggle open the Multilingual Options section to select a multilingual campaign from the dropdown list and to associate import actions through this page with a campaign. You can also click the green + plus sign to add a new campaign name.

After you've confirmed your settings, click Next: Upload file >>.

Upload Your File¶

1 Select the checkbox to confirm that these users have not been purchased or borrowed. You cannot email purchased or borrowed emails through ActionKit. This is important for maintaining a good reputation with ISPs and therefore good deliverability.

2 Before uploading, check the upload summary to verify that the actions will be processed as intended. If an Edit link is present then the value can be changed by configuring the page. The summary also lists the minimum required fields for your file; you might confirm that your file has a value for all required fields.

3 Uncheck the Auto-create user fields if you don't want ActionKit to create new custom user fields for any column with a header beginning user_. If you want to add values to an existing user field, uncheck this box to see a warning and have a chance to correct any misnamed user fields.

4 Click Browse to locate your import file then Start Upload. ActionKit displays your progress during the import.

Note

Uploads can take a while! Before doing a large import, try importing a few rows to make sure the format is correct. You can also compress your file using gzip or zip to reduce upload time.

5 Check for errors or warnings.

• First, ActionKit will check your file format (csv or tsv only, no excel files) and your column headers. You'll be able to correct your headers on the screen directly instead of having to open and edit your file.

• While your upload is running, you'll see a link next to upload status to stop your import.

• When your import is done, you'll see a summary of your upload including the number of rows successfully imported, the number of rows that failed and the number that generated a warning.

• Below the summary, are the details about problem rows. Click on details to see the problem value and the original data from your file.

  Warnings: Warnings tell you that we imported your row but something may be wrong with it. For example, a new US user with zip of "xyzzy" in the file will be created with a blank zip.

  If a row can't be parsed as UTF-8 it's interpreted as Latin-1, which often results in incorrect interpretation of special characters. You can re-import with corrected values or correct the values manually from the individual user record.

  Errors: Rows with errors are not imported. If some rows fail, you can download a csv of the failed rows, making it easier to correct and re-upload only the failed rows.

Import File Requirements¶

Your import file must:

• Be formatted as a TSV or CSV;
• Include a header row with the correct field name(s). If your file contains an unrecognized field name value in the header, you'll see a message and have a chance to correct it.
• Include a column that identifies the user. The choices are email, user_id, akid, or texting_address. ActionKit will match data to existing users' based on this column.

Note

New users cannot be added to ActionKit without an email address, but you can use a fake email.

• Saved in the UTF-8 encoding, if it includes accented characters or international names.
• Specify dates in one of the following formats:

2010-10-28T13:01:59
2010-10-28 13:01:59
2010-10-28 13:01
2010-10-28
12/30/2010 13:01
12/30/2010 13:01:59
12/30/2010
12/30/10 13:01
12/30/10 13:01:59
12/30/10
2010-10-02T13:01:59-0500
2010-10-02 13:01:59-0500

• Specify times in UTC (to see what time it is now in UTC: http://www.time.gov/timezone.cgi?UTC/s/0). The only exception is the last two formats above, which include a time zone offset ("-0500") and can therefore be in any time zone. Formats without a time (ex. "2010-10-28") have the time "00:00:000" assigned.

Skipping Columns In Your File¶

If you want ActionKit to ignore a column, pre-pend the field name with skip_column_. For example, skip_column_1, skip_column_2, etc. This can be helpful when you have a large file to upload and removing a column completely would be difficult but the data is not needed in ActionKit.