InfoConnect for Unisys
Screen Scrape Data
Walkthroughs > Retrieve and Enter Data > Screen Scrape Data

This sample navigates to a screen that has a data table, gets the data line by line, and writes it to a comma separated value list (a .csv file).

You can follow along with the walkthrough and add the code for each step or you can just add the required references and copy in all of the code for the Program.cs file at the end of this walkthrough.

To screen scrape data from a host

  1. In Visual Studio, create a new Console Application project and add references for the following InfoConnect assemblies. (Depending on your version of Visual Studio, these can be found either on the .NET tab or under Assemblies | Extensions.)
    Attachmate.Reflection
    Attachmate.Reflection.Framework
    Attachmate.Reflection.Emulation.IbmHosts
    Attachmate.Reflection.Emulation.OpenSystems

  2. Replace all the code in the Program.cs file with the following code for the terminal you are using.

    This code starts InfoConnect and creates a new demo session. Then it creates an event handler for the NewScreenReady event that will be used to navigate to the appropriate screen and get the data.

    Create a new session with an event handler for the NewScreenReady event
    Copy Code 
    using System;
using System.Collections.Generic;
using System.Text;
using System.IO;
using Attachmate.Reflection.Framework;
using Attachmate.Reflection.Emulation.IbmHosts;
using Attachmate.Reflection.UserInterface;
namespace CreateSessionFromExistingSessionFile
{
    class Program
    {
        
        static void Main(string[] args)
        {
            //Start a visible instance of InfoConnect or get the instance running at the given channel name
            Application app = MyReflection.CreateApplication("myWorkspace", true);

            //Create a new terminal
            IIbmTerminal terminal = (IIbmTerminal)app.CreateControl(new Guid("{09E5A1B4-0BA6-4546-A27D-FE4762B7ACE1}"));

            //Handle the NewScreenReady event to navigate to the screen with the data and then get the data
            IIbmScreen screen = terminal.Screen;
            screen.NewScreenReady += new EventHandler(screen_NewScreenReady);

            //Specify the host name or IP address and connect.
            terminal.HostAddress = "demo:ibm3270.sim";
            terminal.Port = 623;
            terminal.Connect();

            //Make the session visible
            IFrame frame = (IFrame)app.GetObject("Frame");
            frame.CreateView(terminal);
            Console.ReadKey();
        }
    }
}

    This code gets a handle to the terminal of a session running in InfoConnect and then navigates to the screen with the data. It calls the getData() method to screen scrape the data and save it to a comma separated list file.

    Get a handle to a terminal of a running session, navigate to a screen, and call the getTheData method
    Copy Code 
    //ScreenScrapeOS gets the Terminal control of a running session.
//Before you run this sample, make sure the session used in GetControlsByFilePath()
//is running in a InfoConnect workspace.
using System;
using System.Collections.Generic;
using System.Text;
using System.IO;
using Attachmate.Reflection.Framework;
using Attachmate.Reflection.Emulation.OpenSystems;
namespace ScreenScrapeOS
{
    class Program
    {
        static void Main(string[] args)
        {
            //Get a handle to the InfoConnect workspace
            Application app = MyReflection.ActiveApplication;
  
        
            //Make sure InfoConnect is running
            if (app != null)
            {
                //By default, the UserProfile variable is the user's myDocuments folder
                string sessionPath = Environment.GetEnvironmentVariable("UserProfile") + @"\Documents\Micro Focus\InfoConnect\demoSession.rdox";
              
                //Get a handle to the control identified by sessionPath
                object[] terminals = app.GetControlsByFilePath(sessionPath);

                //Make sure the session is running
                if (terminals != null && terminals.Length > 0)
                {
                    //Get the first terminal
                    ITerminal terminal = (ITerminal)terminals[0];

                    //Get a handle to the screen
                    IScreen screen = terminal.Screen;

                    //Go to the screen with the data and wait for it to settle
                    screen.SendKeys("demodata");
                    screen.SendControlKey(ControlKeyCode.Return);
                    screen.WaitForHostSettle(6000, 3000);

                    //Get the screen data and save it to a .csv file
                    getTheData(screen);

                    Console.ReadKey();
                }
                else
                    Console.WriteLine("No such control exists. Check to make sure that the session from the file is running.");
            }
            else
                Console.WriteLine("Failed to get Application object.");
        }
    }
}

     

  3. Add the following code to the Program class in the Program.cs file

    This code navigates to the screen with the data, gets the data and formats it  row by row, using the FormatData helper method. When an empty row is retrieved, the data is saved to a file and written to the Console.

    Handle the NewScreenReady Event to navigate to a screen and  get data
    Copy Code 
     //Navigate to the screen with the data and get the data
        static void screen_NewScreenReady(object sender, EventArgs e)
        {
            string screenID1, screenID2, screenID3;
            IIbmScreen screen = (IIbmScreen)sender;

            screenID1 = screen.GetText(1, 2, 3);
            screenID2 = screen.GetText(1, 1, 5);
            screenID3 = screen.GetText(1, 25, 13);
            if (screenID1 == "ATM")
            {
                //Enter the username and password
                //Use macro.password to put password in
                screen.PutText("username", 20, 16);
                string pWord = screen.Parent.Macro.PasswordBox("Enter anything for a password", "Password");
                screen.PutText(pWord, 21, 16);
                screen.SendControlKey(ControlKeyCode.Transmit);
            }
            else if (screenID2 == "LOGON")
            {
               
                //Go to the screen with the data
                screen.SendKeys("kayak");
                screen.SendControlKey(ControlKeyCode.Transmit);
            }
            else if (screenID3 == "INTERNATIONAL")
            {
                //get the data line by line and format it as a comma separated list
                string data = "";
                string dataLine = "";
                string path = @"c:\temp\data.csv";

                //Find the starting point of the data
                ScreenPoint firstDataPoint = screen.SearchText("Q1", 1, 1, FindOption.Forward);
                int dataRow = firstDataPoint.Row;

                //Adjust column for relative position of Q1 text
                int dataCol = firstDataPoint.Column - 1;

                //Get the text and format it row by row until an empty line is retrieved
                do
                {
                    dataLine = screen.GetText(dataRow, dataCol, 50).Trim();
                    data += FormatData(dataLine) + Environment.NewLine;
                    dataRow++;
                } while (String.IsNullOrWhiteSpace(dataLine) == false);

                //Write the data to the console and a .csv file
                Console.WriteLine(data);
                File.WriteAllText(path, data);
            }
        }

    This code gets the data and formats it  row by row, using the FormatData helper method. When an empty row is retrieved, the data is saved to a file and written to the Console.

    Get the data from the host screen
    Copy Code 
            static void getTheData(IScreen screen)
        {
            string data = "";
            string dataLine = "";
            string path = @"c:\temp\dataOS.csv";//File to write data to

            //Get the screen coordinates of the first row of data
            ScreenPoint startingRow = screen.SearchText("Jan", 1, 1, FindOptions.Forward);
            int dataRow = startingRow.Row;
            int dataCol = startingRow.Column;

            //Get the data line by line and format it as a comma separated list
            do
            {
                //Get a line of data
                dataLine = screen.GetText(dataRow, dataCol, 50).Trim();
               
                //Add the line to the data string
                data += FormatData(dataLine) + Environment.NewLine;
                dataRow++;

                //Continue until there is no data
            } while (String.IsNullOrWhiteSpace(dataLine) == false);
           
           
            //Display the data on the console
            Console.WriteLine(data);

            //Write the data to the comma separated list file
            File.WriteAllText(path, data);
        }
  4. Add the FormatData method to the Program class in the Program.cs file. This method cleans up each row of data and formats it for a comma separated list.
    Format the data for a csv file
    Copy Code 
            public static string FormatData(string sToRemove)
        {
            //remove the extra spaces between numbers
            sToRemove = sToRemove.Replace(" ", "[]");
            sToRemove = sToRemove.Replace("][", "");
            sToRemove = sToRemove.Replace("[]", " ");

            //remove commas in numbers and then replace spaces that act
            //as delimiters with commas to create a comma separated list
            sToRemove = sToRemove.Replace(",", "");
            sToRemove = sToRemove.Replace(" ", ",");
            return sToRemove;
        }
    Format the data for a .csv file
    Copy Code 
            static string FormatData(string dataLine)
        {
            //Replace delimiters with commas and remove the delimiter at the end of the line
            dataLine = dataLine.Replace("|", ",");
            int lastComma = dataLine.LastIndexOf(",") - 1;
            if (lastComma > 0)
            {
                dataLine = dataLine.Substring(0, lastComma);
            }
            return dataLine;
        }

To test this project

  1. If you are using an Open Systems terminal, create a new VT session using a host name of "demo:UNIX" and save it in the default location (...Documents\Micro Focus\InfoConnect) as demoSession.rdox. Then open the session in InfoConnect and log in using any credentials. 
  2. Press F5 to run the project and verify that the data from the session is written to the Console.
  3. Open the c:\temp\dataOS.csv file and verify that the data from the screen is in the file.

 

The full sample for the Program.cs file

This is the full sample for this walkthrough.

Full Code sample for Screen Scrape Data walkthrough
Copy Code 
using System;
using System.Collections.Generic;
using System.Text;
using System.IO;
using Attachmate.Reflection.Framework;
using Attachmate.Reflection.Emulation.IbmHosts;
using Attachmate.Reflection.UserInterface;
namespace CreateSessionFromExistingSessionFile
{
    class Program
    {
       
        //FormatData cleans up the data and formats it as a comma separated list
        public static string FormatData(string sToRemove)
        {
            //remove the extra spaces between numbers
            sToRemove = sToRemove.Replace(" ", "[]");
            sToRemove = sToRemove.Replace("][", "");
            sToRemove = sToRemove.Replace("[]", " ");

            //remove commas in numbers and then replace spaces that act
            //as delimiters with commas to create a comma separated list
            sToRemove = sToRemove.Replace(",", "");
            sToRemove = sToRemove.Replace(" ", ",");
            return sToRemove;
        }

        //Navigate to the screen with the data and get the data
        static void screen_NewScreenReady(object sender, EventArgs e)
        {
            string screenID1, screenID2, screenID3;
            IIbmScreen screen = (IIbmScreen)sender;
            //
            screenID1 = screen.GetText(1, 2, 3);
            screenID2 = screen.GetText(1, 1, 5);
            screenID3 = screen.GetText(1, 25, 13);
            if (screenID1 == "ATM")
            {
                //Enter the username and password
                //Use macro.password to put password in
                screen.PutText("username", 20, 16);
                string pWord = screen.Parent.Macro.PasswordBox("Enter anything for a password", "Password");
                screen.PutText(pWord, 21, 16);
                screen.SendControlKey(ControlKeyCode.Transmit);
            }
            else if (screenID2 == "LOGON")
            {
                //Go to the screen with the data
                screen.SendKeys("kayak");
                screen.SendControlKey(ControlKeyCode.Transmit);
            }
            else if (screenID3 == "INTERNATIONAL")
            {
                //get the data line by line and format it as a comma separated list
                string data = "";
                string dataLine = "";
                string path = @"c:\temp\data.csv";

                //Find the starting point of the data
                ScreenPoint firstDataPoint = screen.SearchText("Q1", 1, 1, FindOption.Forward);
                int dataRow = firstDataPoint.Row;

                //Adjust column for relative position of Q1 text
                int dataCol = firstDataPoint.Column - 1;

                //Get the text and format it row by row until an empty line is retrieved
                do
                {
                    dataLine = screen.GetText(dataRow, dataCol, 50).Trim();
                    data += FormatData(dataLine) + Environment.NewLine;
                    dataRow++;
                } while (String.IsNullOrWhiteSpace(dataLine) == false);

                //Write the data to the console and a .csv file
                Console.WriteLine(data);
                File.WriteAllText(path, data);
            }
        }
        static void Main(string[] args)
        {
            //Start a visible instance of InfoConnect or get the instance running at the given channel name
            Application app = MyReflection.CreateApplication("myWorkspace", true);

            //Create a new terminal
            IIbmTerminal terminal = (IIbmTerminal)app.CreateControl(new Guid("{09E5A1B4-0BA6-4546-A27D-FE4762B7ACE1}"));

            //Handle the NewScreenReady event to navigate to the screen with the data and then get the data
            IIbmScreen screen = terminal.Screen;
            screen.NewScreenReady += new EventHandler(screen_NewScreenReady);

            //Specify the host name or IP address and connect.
            terminal.HostAddress = "demo:ibm3270.sim";
            terminal.Port = 623;
            terminal.Connect();

            //Make the session visible
            IFrame frame = (IFrame)app.GetObject("Frame");
            frame.CreateView(terminal);
            Console.ReadKey();
        }
    }
}
Full Code sample for Screen Scrape Data walkthrough
Copy Code 
//ScreenScrapeOS gets the Terminal control of a running session.
//Before you run this sample, make sure the session used in GetControlsByFilePath()
//is running in a InfoConnect workspace.
using System;
using System.Collections.Generic;
using System.Text;
using System.IO;
using Attachmate.Reflection.Framework;
using Attachmate.Reflection.Emulation.OpenSystems;
namespace ScreenScrapeOS
{
    class Program
    {
        static string FormatData(string dataLine)
        {
            //Replace delimiters with commas and remove the delimiter at the end of the line
            dataLine = dataLine.Replace("|", ",");
            int lastComma = dataLine.LastIndexOf(",") - 1;
            if (lastComma > 0)
            {
                dataLine = dataLine.Substring(0, lastComma);
            }
            return dataLine;
        }
       
        static void getTheData(IScreen screen)
        {
            string data = "";
            string dataLine = "";
            string path = @"c:\temp\dataOS.csv";//File to write data to

            //Get the screen coordinates of the first row of data
            ScreenPoint startingRow = screen.SearchText("Jan", 1, 1, FindOptions.Forward);
            int dataRow = startingRow.Row;
            int dataCol = startingRow.Column;

            //Get the data line by line and format it as a comma separated list
            do
            {
                //Get a line of data
                dataLine = screen.GetText(dataRow, dataCol, 50).Trim();
               
                //Add the line to the data string
                data += FormatData(dataLine) + Environment.NewLine;
                dataRow++;

                //Continue until there is no data
            } while (String.IsNullOrWhiteSpace(dataLine) == false);
           
           
            //Display the data on the console
            Console.WriteLine(data);

            //Write the data to the comma separated list file
            File.WriteAllText(path, data);
        }
        static void Main(string[] args)
        {
            //Get a handle to the InfoConnect workspace
            Application app = MyReflection.ActiveApplication;
  
        
            //Make sure InfoConnect is running
            if (app != null)
            {
                //By default, the UserProfile variable is the user's myDocuments folder
                string sessionPath = Environment.GetEnvironmentVariable("UserProfile") + @"\Documents\Micro Focus\InfoConnect\demoSession.rdox";
              
                //Get a handle to the control identified by sessionPath
                object[] terminals = app.GetControlsByFilePath(sessionPath);

                //Make sure the session is running
                if (terminals != null && terminals.Length > 0)
                {
                    //Get the first terminal
                    ITerminal terminal = (ITerminal)terminals[0];

                    //Get a handle to the screen
                    IScreen screen = terminal.Screen;

                    //Go to the screen with the data and wait for it to settle
                    screen.SendKeys("demodata");
                    screen.SendControlKey(ControlKeyCode.Return);
                    screen.WaitForHostSettle(6000, 3000);

                    //Get the screen data and save it to a .csv file
                    getTheData(screen);

                    Console.ReadKey();
                }
                else
                    Console.WriteLine("No such control exists. Check to make sure that the session from the file is running.");
            }
            else
                Console.WriteLine("Failed to get Application object.");
        }
    }
}