CodeSmith Community
Your Code. Your Way. Faster!

Oracle Schema Provider using System.Data.OracleClient

Latest post 09-05-2007 9:48 PM by vbanda1178. 34 replies.
  • 01-23-2004 5:16 PM In reply to

    • ejsmith
    • Top 10 Contributor
    • Joined on 12-28-2002
    • Dallas, TX USA
    • Posts 2,207
    • Points 1,053,935

    RE: Oracle Schema Provider using System.Data.OracleClient

    VikingCoder,

    Yeah, I'm sorry. :-) I had to change it so that I could get extended properties to be lazy-loadable. All extended properties being loaded was causing big perf problems in databases with a large number of objects.

    Sorry,
    Eric J. Smith

    Eric J. Smith
    CodeSmith Tools, LLC
    Chief Software Architect

    • Post Points: 5
  • 01-23-2004 9:18 PM In reply to

    • gmcelhanon
    • Top 75 Contributor
    • Joined on 05-19-2003
    • Austin, TX
    • Posts 62
    • Points 137,250

    RE: Oracle Schema Provider using System.Data.OracleClient

    The lazy load approach just makes a lot more sense... I'm glad you made that change.
    • Post Points: 5
  • 01-26-2004 4:45 PM In reply to

    • VikingCoder
    • Top 500 Contributor
    • Joined on 11-30-2003
    • Westchester, NY
    • Posts 11
    • Points 720

    RE: Oracle Schema Provider using System.Data.OracleClient

    Agreed... lazy load is way to go!
     
     
    Geoff,
    Have you had a chance to take a look at the new code?
    Comments? Suggestions? Next Steps?

    Post Edited (VikingCoder) : 1/26/2004 4:45:40 PM GMT

    • Post Points: 5
  • 01-28-2004 2:44 AM In reply to

    • gmcelhanon
    • Top 75 Contributor
    • Joined on 05-19-2003
    • Austin, TX
    • Posts 62
    • Points 137,250

    RE: Oracle Schema Provider using System.Data.OracleClient

    Yes, actually, I've done a thorough review of your code.  I'm comfortable with your implementations (some of the code even looks pretty darn familiar ;-) ) and I'm a huge fan of the ALL_ARGUMENTS view.  I wish I had known about that one last Spring!
     
    A couple things I noticed.  I changed your GetViewText implementation back to my original implementation approach which uses the ExecuteScalar method of the OracleCommand object.  The view text is contained within in a single column in a single row, and this is a perfect use for it.
     
    Secondly, the GetCommandResultSchemas implementation isn't what Eric intended for it.  The intention was to return an array of ColumnSchema objects that represented the resultset returned by the command (what I saw you doing was simply returning the return value of the procedure, if there was one).  To do this, we'll need to dynamically construct and execute a command (probably within a transaction - to be safe) to determine what the results look like.  Then from that information, construct the array of ColumnSchema objects and return them.
     
    GetCommandText is a low priority for me as well.  I'd be content just returning the entire package contents for this.  Oracle doesn't split them up and treat them as individual pieces of source code, so why should we feel obligated to do all the hard work? :-)  I can see how it could be useful, but I'm content to leave it alone for now.
     
    I'm also in the process of updating and including my implementation of GetExtendedProperties to take object ownership into account.  This does create a bit of an issue though, since the CODESMITH_EXTENDED_PROPERTIES table needs to be locatable in the database (which leads into my final point).  My initial version used a user centric approach to everything, so the extended properties table was assumed to reside within the current user's schema.  However, by adding CONTAINER_OBJECT_OWNER column to that table, it became clear to me that that approach only made sense if there were multiple schemas using the table to store their metadata in a single CODESMITH_EXTENDED_PROPERTIES table.  There needs to be some way for the user to easily configure whether to use a user-centric approach to everything (with the CODESMITH_EXTENDED_PROPERTIES in each user's schema), or a global approach specifying where the one extended properties table exists (i.e. SYSTEM.CODESMITH_EXTENDED_PROPERTIES).  This leads me to my final point...
     
    And finally, I've started work on adding support for an SchemaExplorer.OracleNativeSchemaProvider.dll.config config file to store some standard appSettings (like for setting the private "mySchemaOnly" variable in the constructor, and a couple of extended properties related values).  Once I am comfortable that my changes are correct, and stable, I'll post the solution back here for you to review.
     
    How's that for feedback?
    • Post Points: 35
  • 01-28-2004 4:00 AM In reply to

    • VikingCoder
    • Top 500 Contributor
    • Joined on 11-30-2003
    • Westchester, NY
    • Posts 11
    • Points 720

    RE: Oracle Schema Provider using System.Data.OracleClient

    You know what they say...Imitation is the sincerest form of flattery.
     
    Wow, you have been a busy little coder!
     
    Things I like, agree with, wish I had thought of...
    • GetViewText as Scalar... nice.
    • SchemaExplorer.OracleNativeSchemaProvider.dll.config config file... Love it! Eric, here's a suggestion... in a future version of CodeSmith you could use Serialization in SchemaProvider for custom props. Maybe even allow for a config dialog that lives in our implementation. 
    • GetExtendedProperties implementation against the CODESMITH_EXTENDED_PROPERTIES with a CONTAINER_OBJECT_OWNER column. Let's just make sure this fails gracefully if the table is absent.
    Things I'm concerned about... (Well there's only one)
    • GetCommandResultSchemas - the idea of creating a command and executing it to determine what the result set looks like will have some problems... What values should I use if the command takes parameters? Plus I'm not all that fond of potentialy changing the DB state during what should be a R/O function... Can you give me a trivial example of a function that you have that returns more than a single value (cursor?) I'd like to ponder this for a while.
    I can't wait to see the changes you make.
    Mike
    • Post Points: 5
  • 01-28-2004 4:25 AM In reply to

    • gmcelhanon
    • Top 75 Contributor
    • Joined on 05-19-2003
    • Austin, TX
    • Posts 62
    • Points 137,250

    RE: Oracle Schema Provider using System.Data.OracleClient

    Currently, the GetExtendedProperties implementation uses the following catch block for the code invoking the ExcuteReader method:
    catch (Exception ex)
    {
        // Check for the Oracle exception that indicates that the table isn't there
        if (ex.Message.IndexOf("ORA-00942") >= 0)
        {
            throw new ApplicationException("Extended properties table 'CODESMITH_EXTENDED_PROPERTIES' does not yet exist.", ex);
        }
        else
        {
            throw ex;
        }
    }
    Maybe a better approach exists, but I think this one is reliable.
     
    For the GetCommandResultSchemas, I can understand your hesitation in executing a command.  One thing to keep in mind is that we can wrap the call up in a transaction, and roll it back when we're done to try and prevent anything serious change to the database (if the command invokes an INSERT command that uses a seqence, the sequence value obviously won't be restored, but for a development environment, this shouldn't be an issue for a properly designed application).  As for what values to pass to the parameters, in the past I've used values unlikely to return any data, like 0's or null values.  This gets you the structure of the results without the data (obviously we should prevent a request for the result schema of a load_all procedure on a large table from locking up your system as it pulls down 1.5 million rows :-) ).  Perhaps Eric can give some insight into the approach he took in his SQL provider.  Eric? Your thoughts?
    • Post Points: 5
  • 01-28-2004 5:44 AM In reply to

    • ejsmith
    • Top 10 Contributor
    • Joined on 12-28-2002
    • Dallas, TX USA
    • Posts 2,207
    • Points 1,053,935

    RE: Oracle Schema Provider using System.Data.OracleClient

    In SQL Server there is a SET FMTONLY option. I set this to on and pass in all NULL values.

    Thanks,
    Eric J. Smith

    Eric J. Smith
    CodeSmith Tools, LLC
    Chief Software Architect

    • Post Points: 5
  • 01-28-2004 7:41 PM In reply to

    • gmcelhanon
    • Top 75 Contributor
    • Joined on 05-19-2003
    • Austin, TX
    • Posts 62
    • Points 137,250

    RE: Oracle Schema Provider using System.Data.OracleClient

    One thing the data environment designer (a relatively obscure tool shipped with VB6 enterprise edition) used to do when you requested this information was pop up an OK/Cancel dialog informing the user that their action could result in modified data.

    Eric, I think you should consider including such a dialog (with a "Go away forever" checkbox feature) wherever you use this particular piece of functionality in your GUIs. To my knowledge, most DBMSs don't give you the ability to get such information in a straightforward manner without actually executing something. While we can certainly wrap the operation up in a transaction for Oracle, it won't prevent the action from causing a change to the state of the server (either by using up the next value of a sequence, which is similar to an IDENTITY column in SQL Server, or if the procedure includes explicit transaction logic).

    Geoff

    Post Edited (gmcelhanon) : 1/28/2004 7:41:47 PM GMT

    • Post Points: 5
  • 02-02-2004 7:01 PM In reply to

    • gmcelhanon
    • Top 75 Contributor
    • Joined on 05-19-2003
    • Austin, TX
    • Posts 62
    • Points 137,250

    RE: Oracle Schema Provider using System.Data.OracleClient

    After spending a good chunk of my evenings over the last week, here is a summary of the changes/additions I have made (beyond those listed above) in no particular order:
     
    I have replaced your GetTableIndexes implementation with my original one (modified to support ownership) which does the same thing, but with a single hit to the database using joins in the metadata query.
     
    I have renamed the extended properties added to the CommandSchema object to be consistent with Eric's convention of "CS_Xxxxx".  The CommandSchema objects returned from the provider will now have the following properties:
    • CS_ObjectType (i.e "PACKAGE")
    • CS_Status (i.e. "VALID", "INVALID")
    • CS_ObjectId (OBJECT_ID of PACKAGE where command resides)
    • CS_OverloadNumber (will be -1 if no overload exists, or a number > 0 if overloads are present)
    • CS_FullName (will contain the original, "unhacked" up name for the command - more on this below).
    I have added support for overloads by reporting the Command.Name property as CUSTOMER.MODIFY[1], CUSTOMER.MODIFY[2], and so on (I thought about using a $ instead of [] to make sure the point was forced syntactically in generated C# code, but in the end decided to go with brackets).  What this means is that in order for you to write templates that will handle overloaded Oracle procedures correctly, you'll need to use the CS_FullName extended property value of the CommandSchema object instead of the Name property.
     
    I have added support for a custom configuration file for use with the DLL.  This is what it looks like:
    <?xml version="1.0" encoding="utf-8" ?>
    <configuration>
        <appSettings>
            <!-- Indicates the schema in which the CODESMITH_EXTENDED_PROPERTIES table should reside.
                    Comment out this entry to always use current user's schema -->
            <!--
            <add key="ExtendedPropertiesTableOwner" value="SYSTEM" />
            -->
       
            <!-- Indicates whether or not the provider should automatically create the extended properties table.
                    Setting this to true creates the table in the schema specified by the 'ExtendedPropertiesTableOwner'
                    value above, or the current user's schema if value isn't present -->
            <add key="AutoCreateExtendedPropertiesTable" value="false" />
       
            <!-- Specifies whether to show user owned objects only. -->
            <add key="ShowMySchemaOnly" value="true" />
           
            <!-- Specifies whether to allow schema provider to execute procedures that return REF CURSORs to determine
                    the column schema information. Allowing this can result in a change in the state of the database due to
                    things like explicit transaction handling in PL/SQL or usage of a sequence's 'nextval' property. -->
            <add key="AllowGetCommandResultSchemas" value="false" />
        </appSettings>
    </configuration>
     
    As you can see, the configuration file contains settings that allow you to control the most critical behavior of the provider.  The user can allow the schema provider to go ahead and create the CODESMITH_EXTENDED_PROPERTIES table (which is excluded from the list returned by GetTables, by the way) if it is not present, and yet still control where it gets created and used from.  It also allows the user to easily filter the schema objects down to just the ones they own directly.  And of course, it allows them to decide whether or not to allow the provider to execute procedures (wrapped in a transactions) to derive the command results.
     
    Also, I cleaned up the existing code a bit.  I've gone to using an approach where all database objects are used within using statements, as this automatically invokes the object's Dispose method upon exit.  Calling Dispose on the Connections, Commands, and DataReaders does everything that Close does and more.  It makes for a more consistent code base, and it's easier to use - the cleanup is done for you automatically.
     
    I have not thoroughly tested all of this new functionality, but I'm comfortable with the results I've seen in my first pass of testing.  I will post the source later when I have time.  Right now I've got to get some real work done!
     
    Cheers,
    Geoff
     
    • Post Points: 5
  • 02-02-2004 8:31 PM In reply to

    • VikingCoder
    • Top 500 Contributor
    • Joined on 11-30-2003
    • Westchester, NY
    • Posts 11
    • Points 720

    RE: Oracle Schema Provider using System.Data.OracleClient

    Can't wait to see all the good work.
    Mike
    • Post Points: 5
  • 02-03-2004 3:50 AM In reply to

    • gmcelhanon
    • Top 75 Contributor
    • Joined on 05-19-2003
    • Austin, TX
    • Posts 62
    • Points 137,250

    RE: Oracle Schema Provider using System.Data.OracleClient

    To "install" the provider, just copy the following files to your C:\Program Files\CodSmith\v2.5 (or equivalent) folder:
    • SchemaExplorer.OracleNativeSchemaProvider.dll
    • SchemaExplorer.OracleNativeSchemaProvider.dll.config
    Mike, have a look at the source and let me know if you have any ideas for further improvements.  I'm pretty happy with what I see.  There are still a few obscure problems that I'm aware of, but not too concerned about (like a procedure I have that updates a record, but returns before and after REF CURSORs fails to provide column schema information -- but regular SELECT procedures do just fine).

    Geoff
     
    Revision History
    2/2/2003 - First release.
    2/3/2003 - Fixed a bug in GetCommandParameters related to procedure overload handling.

    Post Edited (gmcelhanon) : 2/4/2004 3:43:13 AM GMT

    • Post Points: 5
  • 05-10-2004 4:14 AM In reply to

    • fevets
    • Not Ranked
    • Joined on 03-25-2004
    • Posts 2
    • Points 50

    RE: Oracle Schema Provider using System.Data.OracleClient

    I am using the Oracle provider. I am no Oracle expert, but my Oracle database has 2 tables (it has alot more but for purpose of this it is 2) table A and B. I defined Primary Keys for both tables and table B has a Foreign key to table A's Primary Key. All fine so far. My CodeSmith template is iterating the tables in the database and exporting the schema to an XML file. When I iterate table B's ForeignKeys collection it returns the correct data and is ok, however iterating table A's PrimaryKeys collection returned nothing. I was expecting it to return a collection with a single TableKeySchema instance for the key relating it to table B.

    The SchemaAPI doco says what I thought should happen but it does not work as I expected.cry

    ideaSo, I modded the Oracle provider to function as I expected and it now works. The change is very simple, I have highlighted changed/new bits below ..

    public TableKeySchema[] GetTableKeys(string connectionString, TableSchema table)
            {
                ArrayList tableSchemaList = new ArrayList();

                using (OracleConnection conn = new OracleConnection(connectionString))
                {
                    // Open a connection
                    conn.Open();

                    string sql = string.Format( 
                        @"select 
                        cols.constraint_name, 
                        cols.column_name, 
                        cols.position, 
                        cons.table_name,
                        r_cons.table_name related_table_name, 
                        r_cols.column_name related_column_name 
                    from
                        all_constraints     cons,
                        all_cons_columns    cols,
                        all_constraints     r_cons,
                        all_cons_columns    r_cols
                    where cons.OWNER = '{0}'
                      and (cons.table_name = '{1}' or r_cons.table_name = '{1}')
                      and cons.constraint_type='R'
                      and cols.owner = cons.owner
                      and cols.table_name = cons.table_name   
                      and cols.constraint_name = cons.constraint_name 
                      and r_cols.owner = cons.r_owner 
                      and r_cols.constraint_name = cons.r_constraint_name 
                      and r_cons.owner = r_cols.owner 
                      and r_cons.table_name = r_cols.table_name 
                      and r_cons.constraint_name = r_cols.constraint_name 
                    order by cons.constraint_name, cols.position",
                        table.Owner,
                        table.Name);

                    // GKM - Should we change to OracleDataReader implementation?
                    OracleDataAdapter da = new OracleDataAdapter( sql, conn ); 

                    DataSet ds = new DataSet();
                    using (ds)
                    {
                        da.Fill( ds );

                        string lastConstraint = string.Empty;
                        foreach (DataRow row in ds.Tables[0].Rows )
                        {
                            if (lastConstraint!=row["constraint_name"].ToString())
                            {
                                lastConstraint=row["constraint_name"].ToString();

                                string pkTable = row["related_table_name"].ToString();

                                DataRow [] distinctColumns = ds.Tables[0].Select(
                                    string.Format("constraint_name = '{0}'", lastConstraint) ); 

                                ArrayList fkMemberCols = new ArrayList();
                                ArrayList pkMemberCols = new ArrayList();

                                foreach( DataRow rdc in distinctColumns)
                                {
                                    fkMemberCols.Add( rdc["column_name"].ToString());
                                    pkMemberCols.Add( rdc["related_column_name"].ToString());
                                }

                                TableKeySchema tks = null;
                                if (pkTable != table.Name)
                                {

                                    tks = new TableKeySchema( 
                                        table.Database,
                                        lastConstraint,
                                        (string[])fkMemberCols.ToArray(typeof(string)),
                                        table.Name,
                                        (string[])pkMemberCols.ToArray(typeof(string)),
                                        pkTable );
                                }
                                else
                                {
                                    tks = new TableKeySchema( 
                                        table.Database,
                                        lastConstraint,
                                        (string[])fkMemberCols.ToArray(typeof(string)),
                                        row["table_name"].ToString(),
                                        (string[])pkMemberCols.ToArray(typeof(string)),
                                        pkTable );
                                }

                                // Don't forget to add it to our result
                                tableSchemaList.Add( tks );
                            }
                        }

                    }
                }
                
                return (TableKeySchema[]) tableSchemaList.ToArray(typeof(TableKeySchema));
            }


    Can someone let me know if this is the correct way to get the PrimaryKeys collection working or is there a less hacky way of doing it.

    Cheers

    fevets
    • Post Points: 45
  • 05-12-2004 8:03 PM In reply to

    • gmcelhanon
    • Top 75 Contributor
    • Joined on 05-19-2003
    • Austin, TX
    • Posts 62
    • Points 137,250

    RE: Oracle Schema Provider using System.Data.OracleClient

    First thing is that this thread actually contains an older version of the schema provider. I'd recommend using the latest version.

    Second thing is that I had actually found the problem you described above, and it was a misunderstanding on my part about how the SchemaExplorer library worked. I originally thought it would infer the relationships from the Foreign Keys.

    I have fixed this problem in the latest version of the schema provider, but I don't think I've uploaded it to the other thread (the "Unofficial" Oracle provider). I should do that!
    • Post Points: 5
  • 08-30-2004 6:23 AM In reply to

    • kishorda
    • Not Ranked
    • Joined on 08-30-2004
    • Posts 1
    • Points 5

    RE: Oracle Schema Provider using System.Data.OracleClient

    Can someone help me in using this SchemaExplorer.OracleNativeSchemaProvider.dll. As i have added the file at the given location what is the next step such that codesmith recognises it.
    • Post Points: 5
  • 12-12-2004 12:40 AM In reply to

    • BillS
    • Top 500 Contributor
    • Joined on 07-07-2003
    • Posts 16
    • Points 305

    RE: Oracle Schema Provider using System.Data.OracleClient

    Installed 2.6, copied the following from the 2.5 directory to the 2.6, but when I go to add a datasource from tablepicker, the oracleschemaprovider is not displayed. Using process explorer I see that codesmith is attached to v1.0.1494.38426 of schemaexplorer.oraclenativeschemaprovider.dll

    SchemaExplorer.OracleNativeSchemaProvider.dll.config
    SchemaExplorer.OracleNativeSchemaProvider.dll

    Is there a way to diagnose this? I rebooted after the install.
    • Post Points: 5
Page 2 of 3 (35 items) < Previous 1 2 3 Next > | RSS
Copyright © 2008 CodeSmith Tools, LLC
Powered by Community Server (Commercial Edition), by Telligent Systems