FAQ

Why should I use DataMine?

DataMine is ideal for extracting and analyzing all of your raw and select processed data to create custom metrics which can be exported or visualized using custom dashboards. It's perfect for gaining deeper insights into user behaviors.

What audience is this tool for?

DataMine is a powerful tool for social and mobile app marketers, developers, product managers, business intelligence analysts, and anyone familiar with SQL or other database querying languages looking for custom insights. However, even if you have limited technical knowledge, DataMine was designed to have its results easily accessible across the board.

What technologies and/or languages do I need to know in order to use DataMine?

Users should be familiar with writing queries in HiveQL, SQL or any other database querying languages. It's also helpful to know the custom events in your apps.

How do I access DataMine?

To utilize custom metrics and DataMine, speak to your CSM today.

What instrumentation is required?

There is no additional instrumentation required for DataMine. If your app is instrumented correctly, all the data that is required to use this tool is already available. This means you can start using DataMine as soon as you integrate Upsight.

What technologies or languages do I need to know in order to use DataMine? Users should be familiar with writing queries in HiveQL, SQL or any other database querying languages. It's also helpful to know the custom events in your apps.

What affects my query speed?

• Upsight DataMine is a shared resource - In order to process the huge amounts of data collected by Upsight, we have a tremendous cluster (to emphasize: it's really big) for our customers to tap into. Processing requests sent to this cluster are balanced so that each of our customers get a "fair share" of the resources.
  • If the cluster happens to be idle, you may receive far more than your normal fair share and your query will run quickly.
  • During periods of higher utilization, resources need to be divided amongst all customers and queries may complete less quickly.
• Query complexity - if you are looking at large windows of data, or are executing a query that has many steps, these queries will take longer to complete than a more simple query.
  • Pro Tip : Always try to filter by month/day/hour, application, country, operating system, etc. as this will reduce the scope of work the query has to do.
• Query setup & execution - Queries submitted through Upsight have to go through a series of steps in order to produce results. The setup of a query and returning of results can add some additional time to your query speed, depending on query complexity.

How can I make my queries run faster? Why do even simple queries take 5-10 minutes to run?

When a query is executed, it enters a queue where it waits for available resources on a shared Hadoop cluster. It then goes through a “map and reduce” setup job that crunches large data before it runs. Therefore while a simple query might have a short runtime, it will spend time in the queue and job setup process. However, because our system is built to scale for big data, runtime will not multiply even as your data multiplies. Therefore, a simple million-row query can be returned in about the same time it takes for the same query run for a hundred rows.

To improve the speed of queries, see Best Practices for tips on writing efficient queries. Upsight also offers the option to purchase dedicated resource lanes for your queries to run in a private queue.

Why am I missing tables?

Your CSMs will enable table access in DataMine. Some tables require specific messages to be triggered before they will be created. See Data Schema Tables for more information.

Why did my DataMine query fail?

If your query has failed, DataMine will display a error message. If it is HQL related, fix the query and submit again. Other error messages include:

• “An Unexpected Error has occurred.”

An error occurred with submitting the query. Common causes are validation time out or the user navigating away from the DataMine UI before the query fully posted. Try submitting the query again. If the error persists, write in to support@upsight.

• “Query has failed.”

The query failed in DataMine. A common cause is a query hitting out of memory issues due to its size. If you are running a large query, try reducing the amount of days and data queried. Other times it is an issue with the execution engine. Write in to support@upsight and include information about your failed query.

I want to analyze campaign results / user drop off / revenue etc. Can you assist in writing the query?

The Upsight Professional Services team provides query writing and analytics support. Contact your CSM for more information.