QCollector Data Interface
Revision 2.11
November 5, 2011

Contents

  1. Introduction
  2. Requirements
  3. Overview
  4. Information Request Messages
  5. Command Messages
  6. Real-time File Update Messages
  7. Real-time Last Record Update Messages

Introduction

QCollector Expert has a built-in data interface that can be used by other Windows programs to:

  • Query QCollector for information
  • Instruct QCollector to retrieve history data from the servers
  • Manage portfolios
  • Be notified when QCollector updates data files in real-time.
All versions of QCollector support the QCollector Data Interface.

Traders and programmers may find it simpler to use the QCollector Data Interface rather than writing their own code to connect with a data feed service and manage and store data retrieved from the service. QCollector provides the same interface for all supported data feeds so only one code base needs to be managed by the developer, even if the data feed service changes. Currently eSignal, QCharts, and DTN IQFeed are supported.

Requirements

  • QCollector Expert, which can always be downloaded from the Tradeworks Software web site. The QCollector Data Interface is accessible with or without a license key installed.
  • A subscription to one of the supported data feed services: eSignal, QCharts, and DTN IQFeed are currently supported.
  • Experience developing Windows applications. The QCollector Data Interface is language independent but you must be able to write code to send and receive Windows messages.

Overview

The QCollector Data Interface relies on Windows messages and the Windows Global Atom Table to accept commands and exchange information with client programs. Atoms are references to text strings, managed by Windows, which can be used by Windows programs to share information.

The messages in the QCollector Data Interface can be grouped as follows:

  1. Information Requests - Client applications can query QCollector for information about QCollector portfolios, the items in the portfolios, and the items being updated in real-time in the QCollector real-time udpates list.

  2. Commands - Command messages can be used to:
    • Instruct QCollector to add a new symbol/bar interval item to a portfolio
    • Download history data and use it to create or update a data file in a portfolio
    • Delete items from a portfolio, and, optionally, their data files as well
    • Add or remove portfolio items from from the real-time udpates list
    • Clear all items from the real-time udpates list.

  3. File Update Notification Messages - These messages are used by QCollector to notify client applications when data files have been updated on a QCollector real-time update cycle. Each message will include information about the individual data files which were updated such as the symbol associated with the data file and the data file's fully qualified path and file name. Applications send a subscription message to QCollector when they want to receive file update notification messages. When a client application receives the update messages it can read the refreshed data file to get the latest real-time price and trade information for that ticker symbol.

  4. Last Record Update Notification Messages - QCollector uses this group of messages to send the last record from each data file that was updated on a QCollector timed update cycle. Unlike the file update notification messages described above, the last record update notification messages send the actual price data from the data file. The message will include the symbol, interval, file name and the complete last record (i.e. O,H,L,C,V) from the updated data file. Client applications send a subscription message to QCollector when they want to receive Last Record Update Notification messages.

Registering Messages

Before accessing the QCollector Data Interface, client applications must first register the predefined interface messages with Windows by calling the Windows RegisterWindowMessage function. Pass the text definition for the message you want to use and then store the returned value in a variable. For example:

wm_QCollectorClientDataRequest := RegisterWindowMessage('QCOLLECTOR_CLIENT_DATA_REQUEST');

QCollector also registers these messages with Windows.

Note: In this document the messages defined in the QCollector Data Interface are highlighted in blue and printed as the actual text that is passed to the RegisterWindowMessage function. For example QCOLLECTOR_CLIENT_DATA_REQUEST.)

You only need to register the messages your program will actually be using.

QCollector Data Interface Window Handle

Before sending messages to QCollector you must retrieve the QCollector window handle used for the Data Interface. You can do this by searching for the window class name using the Windows API "FindWindow" function.

For QCollector Expert the window class name used for the data interface is QCDataInterfaceWndClass.

For earlier versions of QCollector the data interface window class name is TQCollectorMain.

Note: For backward compatibility QCollector Expert has an option in the Program Settings to use the previous version data interface window class name.

Example Finding the Data Interface window handle to use when sending messages to QCollector Expert:

Hwnd := FindWindow('QCDataInterfaceWndClass', nil);

Information Request Messages

 

QCOLLECTOR_CLIENT_PORTFOLIO_LIST_REQUEST - Send this message to request a list of the QCollector portfolios. When QCollector receives this message it will save a list of portfolio information to a data file (see example below) in the QCollector application data folder. Set lParam to the window handle (HWnd) where you want the portfolio list request complete message to be sent. wParam is not used.

QCOLLECTOR_PORTFOLIO_LIST_REQUEST_COMPLETE - After QCollector saves the portfolio list to a temp file, it will will send this message to the HWnd that requested the portfolio list. lParam is an Atom holding one string value which is the full path and file name for the portfolio list file. QCollector deletes the Atom when SendMessage returns. On receipt of this message the client application can read the file to get the list of current portfolios in QCollector.

The data in the portfolio list file saved by QCollector is plain text with one line for each portfolio formatted as follows:

{QCollector portfolio name}|{Format char['A' for ASCII, 'M' for MetaStock]}|{Data folder path}

Each field value is separated with the vertical bar ("|") character. Example:

My Stocks|A|C:\Data\MyStocks
MetaStock1|M|C:\MsData\MetaStock1

QCOLLECTOR_CLIENT_PORTFOLIO_ITEMS_REQUEST Send this message to QCollector to retrieve a list of all items in a QCollector portfolio. The message wParam must hold an Atom with the portfolio name (or portfolio data folder path) for the QCollector portfolio for which you want to retrieve the item list. Set lParam to the window handle (HWnd) where you want the portfolio item list request complete message sent.

When QCollector receives this message it will create a list of all items in the named portfolio and save it to a text file. Each line in the text file will include the symbol, interval, description, and data file name for one item.

QCOLLECTOR_PORTFOLIO_ITEMS_REQUEST_COMPLETE - QCollector sends this message to the window handle that requested the portfolio items list. lParam is an Atom holding one string value - the full path and file name for the portfolio item list text file created by QCollector. QCollector deletes the Atom when SendMessage returns.

The data in the portfolio item list file is plain text with one line for each item formatted as follows:

{symbol}|{(interval)}|{title}|{data file name}|{spare(0)}
Symbol - for that item.
Interval - D, W, M, Y for Daily, Weekly, Monthly, and if applicable, Yearly, or a number for intraday minutes.
Title - Company name or description for the item.
Data file name - the name of the item data file. (No path, data files are always stored in the portfolio folder.)
Spare - for future use.

Each field value is separated with the vertical bar ("|") character. An example portfolio item list:

IBM|15|International Business Machines|IBM_15.csv|0
MSFT|1|Microsoft Corporation|MSFT_1.csv|0
ADBE|DAILY|Adobe Systems Incorporated|ADBE_D.csv|0

QCOLLECTOR_CLIENT_REALTIME_SYMBOL_LIST_REQUEST - Send this message to retrieve a list of all items in the QCollector real-time updates list. When QCollector receives this message it will save the symbol, interval and title for each item to a file in the QCollector application data folder. lParam holds the window handle where you want the real time symbol list request complete message sent. wParam is not used. Your application can read the information from this file when it receives the real-time symbol list request complete message below.

QCOLLECTOR_REALTIME_SYMBOL_LIST_REQUEST_COMPLETE - QCollector sends this message to the window handle that requested the list. lParam is an Atom holding one string value - the full path and file name for the list text file. QCollector deletes the Atom when SendMessage returns. On receipt of this message your application can read the file to get information about all the items in the QCollector timed updates list.

The data in the text file saved by QCollector has one line for each item formatted as follows:

{symbol}|{interval)}|{title}

Each field value is separated with the vertical bar ("|") character. For example:

ADBE|5|Adobe Systems Incorporated
INTC|D|Intel Corporation

Command Messages

 

Command messages can be sent to QCollector to have QCollector perform specific actions.

Client data request messages are sent to QCollector to retrieve data for a symbol/bar interval data series. Each data request must include the name of a target portfolio (or portfolio folder path).

When QCollector recieves a client data request it will first search the target portfolio for any item matching the symbol/bar interval named in the request. If no item is found a new item will be created. If a matching item is found in the target portfolio QCollector will simply bring the data file for the existing item up to date and then send the data request complete message.

When adding a new item to a portfolio, QCollector looks for any saved default portfolio item properties, and if found, will use them to create a new item. If the user hasn't saved any default item properties the item will be added to a portfolio with the default portfolio properties. Saved default properties can include whether or not to store volume or open interest in the data file, start and end session times, etc. In most cases this will not change any essential formatting of the data in the data files.

QCOLLECTOR_CLIENT_DATA_REQUEST - Send this message to QCollector to request data for a symbol. wParam must hold the Atom with the formatted data request string. Set lParam to the HWnd where you want the data request complete message to be sent.

The data request string must be formatted as follows, with no line breaks:

{ID string}|{symbol}|{bar interval}|{days back}|{collect now [0,1]}|
{add to timed updates list [0,1]}|{spare(0)}|{portfolio name or portfolio folder path}
ID string - user defined. This string will be returned with the data request complete message. Your application can use this value to track multiple data requests if required. Set to '0' if not required.
Symbol - ticker symbol.
Bar interval - D, W, M for Daily, Weekly, or Monthly, or set a number for intraday minute bars. Use 0 for tick data series.
Days back - calendar days back to set first date for data file. Set this value to zero to start at beginning of current day.
Collect now - Downloading data is optional with client data requests. Set this value to 1 to collect data for the item immediately, else 0. If set to zero then a new item will simply be added to the QCollector portfolio if a matching item isn't found. Multiple new items may be added to a portfolio this way, without collecting data immediately.
Timed updates - Set to 1 to add the item to the QCollector real-time updates list, else set to 0. If set to 1 and the QCollector real-time updates are running when the request is received, the item's data file will be updated in real-time along with the rest of the items in the list.
Spare - (0) for future use.
Portfolio name - the name or target portfolio, or, the path to the portfolio data folder.

Each field value in the request string must be separated with the vertical bar ("|") character.

Notes

  • If a portfolio item matching the request ticker symbol/bar interval combination is found in the portfolio, the found item will be used for the data request. If no matching item is found in the portfolio, a new item will be created using any saved default properties for portfolio items.
  • When sending an Atom to QCollector always use SendMessage, never PostMessage. Delete the Atom when SendMessage returns.
  • The data request can fail for a number of reasons, such as syntax errors, invalid ticker symbols, etc. Always check the SendMessage return value to see if the data request was handled successfully.

Return Values

SendMessage will return 1 if QCollector successfully received and parsed the message. The data request has been sent to the servers and your app will receive the data request complete message when the data is ready.
Returns -1 if there was a problem with the format of the request string. No data request complete message will be sent back to your application. Further information about the problem may be found in the QCollector Event Log.
Returns 0 if QCollector never received the message. No data request complete message will be sent back to your application.

Example 1
Details: Request daily data for IBM, going back 100 days, in portfolio named "My Stocks". Collect the data immediately and do not add item to timed updates list. 

   0|IBM|D|100|1|0|0|My Stocks

Example 2
Details: Request 5 minute intraday data for IBM, going back 10 days. Collect the data immediately and save in portfolio named "My Stocks". Add the portfolio item to the real-time updates list. Use a custom reference of "0148855".

  0148855|IBM|5|10|1|1|0|My Stocks

QCOLLECTOR_DATA_REQUEST_COMPLETE - QCollector will send this message to the client application when a previously submitted data request is complete, meaning the data file has been updated and is ready for reading. lParam is the Atom with string information for the previous request. wParam is not used.

The Atom data string will be formatted as follows:

{ID string}|{full path and file name to the data file}
ID string - Optional user defined custom identifier which was sent with the original data request.
Full path and file name to the updated data file in the portfolio folder specified by the client data request.

Example Data Request Complete Atom

  0148855|C:\Data\Stocks\IBM_5.csv

Note: You should not assume there will be data in the data file, or even that the data file will exist. The response from the server may be "No Data" for any number of reasons. Further information about failed data requests may sometimes be found in the QCollector Event Log.

QCOLLECTOR_CLIENT_REMOVE_REALTIME_SYMBOL_REQUEST - Send this message to QCollector to have an item removed from the QCollector real-time updates list. wParam must hold the Atom with the formatted request string. There is no "request complete" message so lParam is not used. SendMessage will return 1 if the request succeeds.

The request string must be formatted as follows:

{symbol}|{interval}
Symbol - Ticker symbol
Interval - D, W, M for Daily, Weekly, or Monthly, or a number for intraday minutes. Use 0 for tick data items.

When QCollector receives this message it will search through the items in the real-time updates list and remove the first one matching the symbol/bar interval combination.

QCOLLECTOR_DELETE_PORTFOLIO_ITEM - Send this message to have QCollector delete an item from a portfolio. wParam must hold the Atom with the formatted request string. No message is returned from QCollector so lParam is not used. SendMessage will return 1 if the request succeeds.

Note: Requires QCollector Expert version 4 or later.

The request string to delete a portfolio item must be formatted as follows:

{Symbol}|{Bar interval}|{Delete file [1,0]}|{Portfolio name}
Symbol - the ticker symbol.
Bar interval - D, W, M for Daily, Weekly, or Monthly, or a number for intraday minute bars. Use 0 for tick data items.
Delete file - 1 to delete any data file associated with the item, else 0.
Portfolio name - the name of the portfolio to search for a matching item to delete.

Example

  IBM|D|1|My Stocks

When QCollector Expert receives this message it will check the named portfolio for the first item matching the symbol/bar interval combination, and, if found, it will delete the item from the portfolio. If the "delete file" parameter is set to "1", the data file for the found item will be deleted when the item is deleted from the portfolio.

Note: The "delete file" parameter only applies to ASCII format portfolios. MetaStock format portfolios will always delete the data file.

QCOLLECTOR_CLIENT_CLEAR_REALTIME_SYMBOLS - Send this message to QCollector to clear all items from the QCollector real-time updates list. There is no request complete message. SendMessage will return 1 if the request succeeds.

Real-time File Update Notification Messages

  These messages are used to receive notifications when files are updated with real-time streaming data using the QCollector Expert Real-time Updates feature.

QCOLLECTOR_REGISTER_FOR_FILE_UPDATES - Send this message to QCollector to register to receive the file update notification messages. lParam holds the window handle you want QCollector to send the messages to.

QCOLLECTOR_UNREGISTER_FOR_FILE_UPDATES - Send this message to QCollector when you no longer want to receive file update notification messages. lParam holds the window handle that QCollector will unregister.

QCOLLECTOR_FILE_UPDATE - After all data files have been updated on a timed update cycle QCollector will send this message out once for each file that was updated. lParam is a handle to a Global Atom that contains info in text format about the file that was updated. The Atom text string includes the symbol, the record interval, and the fully qualified path and data file name for the security that was updated. A comma and a semi-colon are used as separator characters. Your application can parse this string to get the separate pieces of information. See the examples following.

QCOLLECTOR_ALL_FILES_UPDATED - QCollector sends one message to each registered client application after all data files have been updated in the timed update cycle, indicating that the cycle is complete. lParam holds the number of data files that were updated.

Example 1: An ASCII format data file named "MSFT_5.CSV" in folder "C:\Data\Ascii\Stocks\" stores data for symbol "MSFT" in records with 5 minute bars. When the file is updated on a timed update cycle the text in the global Atom sent by QCollector will be "MSFT,5;C:\Data\Stocks\MSFT_5.CSV"

Example 2: A MetaStock format data file in a folder named "C:\Data\MS\Stocks\" stores data for a security with symbol "ADBE". The data is stored in a file named "F6.dat" and the record interval is Daily. The text in the global Atom sent by QCollector will be "ADBE,D;C:\Data\MS\Stocks\F6.dat".

Important! After receiving these messages, do not delete the Atom by calling GlobalDeleteAtom; QCollector will do that when the message handling function returns.

Real-time Last Record Update Notification Messages

  These messages are used to receive notifications when files are updated with real-time streaming data using the QCollector Timed Updates feature, and to receive the last record from each udpated data file.

QCOLLECTOR_REGISTER_FOR_LAST_RECORD_UPDATE - Sends this message to QCollector to register to receive the last record update notification messages. lParam holds the window handle you want QCollector to send the messages to.

QCOLLECTOR_UNREGISTER_FOR_LAST_RECORD_UPDATE - Send this message to QCollector when you want to stop receiving the last record update notification messages. lParam holds the window handle to unregister.

QCOLLECTOR_LAST_RECORD_UPDATE - This message is sent once for each updated data file on completion of a timed update cycle. The message lParam is an Atom which holds with the actual last bar record from the data file, in plain text.

For all data files this text string begins with the symbol, the interval, and the data file name (no path) all delimited with tab characters. For MetaStock format data files the rest of the string is the date, time, open, high, low, close and volume values of the last data record with all fields again delimited by tab characters. For ASCII format data files the rest of the string is the exact text of the last record in the ASCII data file which will include whatever delimiter characters are specified for use in that particular portfolio.

Example
QCollector is updating a data file named "IBM_5.CSV" on a timed updates cycle. The QCOLLECTOR_LAST_RECORD_UPDATE message is sent to subscribed applications each time the data file is updated on the timed update cycle. The text contained in the global Atom sent with the message is 

"IBM{tab}5{tab}IBM_5.CSV{tab}05/19/2002,10:30:00,112,112.5,111.5,112.25,23500"
with the price record changing on each update.

Daily data series will have a "D" in the second field, weekly will have a "W", monthly will be "M", yearly will be "Y".

Important: For MetaStock format data files the date format used in the QCOLLECTOR_LAST_RECORD_UPDATE message will be the Short Date Format set in the user's Windows Regional settings. The time format will always be "hh:mm:ss" without regard to the user's regional settings. Also, for MetaStock format, if the data series interval is "Daily" or greater, there will still be a time field, which will be set to "00:00:00" and can be ignored.

Important: Do not delete the Windows Global Atom by calling GlobalDeleteAtom; QCollector will do that when the message handling function returns.

Return to Top

Copyright © 2000-2011 Tradeworks Software