cjhaas.com Basically a place that Chris can post solutions to problems so he can easily find them later

February 2, 2011

Don’t bother with TcpClient.Connected

Filed under: Uncategorized — Tags: — Chris Haas @ 2:38 pm

The TcpClient has a Connected property that is very convenient to use but unfortunately it doesn’t do what you think it should do. A better name for this property would be WasConnected or WasLastOperationSuccessful. The problem is that this property only tells you the status of the last operation. For example, if 30 seconds ago you sent some data this property would be true. If the client that you sent data to calls Close() on their end or their network connection goes down this property will still be true. The latter case you can probably understand but the for the former case you need to understand that there’s no “connection agreement” between the two parties. When one side calls Close() its not going to stay open to send data to the other and potentially wait forever on a slow network connection. Instead, Close() just means “terminate my side”. If you want, you can roll your own handshake implementation and have the client send a ‘closing connection’ packet but neither side should assume that it will work.

For more information see MSDN:

Because the Connected property only reflects the state of the connection as of the most recent operation, you should attempt to send or receive a message to determine the current state. After the message send fails, this property no longer returns true. Note that this behavior is by design. You cannot reliably test the state of the connection because, in the time between the test and a send/receive, the connection could have been lost. Your code should assume the socket is connected, and gracefully handle failed transmissions.

December 14, 2010

How to group items in InDesign using VB.Net/C#

Filed under: Uncategorized — Tags: , — Chris Haas @ 12:12 pm

First off, give up going the native route. There’s no way that I’ve found that you can convert an array to an InDesign.Objects object. Also, subclassing InDesign.Objects doesn’t appear to work either. What does ultimately work? JavaScript. Don’t worry, you can create JS on the fly and have InDesign execute it for you using the DoScript method of the Application object.

Below is an extension method of the InDesign.Document object that takes two or more InDesign.PageItem Id‘s. I haven’t found a way to cast specific objects (InDesign.Rectangle, InDesign.TextFrame, etc) to generic InDesign.PageItem so instead of writing tons of method signatures I’m just passing their internal integer IDs. The code is in VB.Net and should be pretty self-explanatory so someone should be able to convert it to C# very easily. Yes, I know the newlines aren’t necessary, but I like them. Why an Extension method? This snippet is one of many in my library and I find them easier to use if they’re extensions instead of sitting in a helper class.


    <Extension()>
    Public Function GroupObjects(ByVal indesignDocument As InDesign.Document, ByVal ParamArray objectIds() As Integer) As InDesign.Group
        'Sanity checks
        If indesignDocument Is Nothing Then Throw New ArgumentNullException("indesignDocument")
        If objectIds Is Nothing OrElse objectIds.Count < 2 Then Throw New ArgumentException("You must pass at least 2 object ids")

        'We'll assign a unique label to the group that we create in JavaScript so that we can find it in managed code later
        Dim GID = Guid.NewGuid().ToString()

        'Create the JavaScript
        Dim Buf As New StringBuilder()
        Buf.AppendLine("var items = new Array();")
        For Each ID In objectIds
            Buf.AppendFormat("items.push(app.activeWindow.activePage.pageItems.itemByID({0}));", ID)
            Buf.AppendLine()
        Next
        Buf.AppendLine("var g = app.activeWindow.activePage.groups.add(items);")
        Buf.AppendFormat("g.label='{0}';", GID)

        Dim IA = indesignDocument.Parent
        IA.DoScript(Buf.ToString(), InDesign.idScriptLanguage.idJavascript)

        'Loop through all document groups looking for the object with the label created above
        For Each G As InDesign.Group In indesignDocument.Groups
            If Not String.IsNullOrWhiteSpace(G.Label) AndAlso G.Label = GID Then Return G
        Next
        Return Nothing
    End Function

To call this Extension method, assuming that you have an InDesign.Document object called ID, a TextFrame called TF and a Rectangle called R:

ID.GroupObjects(TF.Id, R.Id)

Just to help people that are searching, one of the error messages that this will help resolve is:
Unable to cast object of type 'System.Object[]' to type 'InDesign.Objects'.

October 6, 2010

Cannot access a closed Stream with MemoryStream and StreamWriter

Filed under: Uncategorized — Tags: , — Chris Haas @ 8:53 am

A StreamWriter will close the underlying stream when Dispose is called so if you try the following pattern you’ll receive a “Cannot access a closed Stream” error message.

        Using MS As New MemoryStream()
            Using SW As New StreamWriter(MS, System.Text.Encoding.UTF8)
                SW.Write("Hello")
            End Using

            Using SR As New StreamReader(MS, System.Text.Encoding.UTF8)
                Return SR.ReadToEnd()
            End Using
        End Using

To fix this, you need to find a way to use your StreamReader before calling the StreamWriter‘s Dispose (which End Using does). You also need to make sure that you reset the stream’s current Position to the beginning of the stream. And while I don’t think that a MemoryStream buffers I recommend calling Flush on the StreamWriter just in case.

        Using MS As New MemoryStream()
            Using SW As New StreamWriter(MS, System.Text.Encoding.UTF8)
                SW.Write("Hello")
                'Flush any buffers
                SW.Flush()

                'Reset the stream position
                MS.Position = 0

                'Create and use the StreamReader before calling the StreamWriter's Dispose()
                Using SR As New StreamReader(MS, System.Text.Encoding.UTF8)
                    Return SR.ReadToEnd()
                End Using
            End Using
        End Using

August 12, 2010

“Could not find any resources appropriate for the specified culture or the neutral culture”

Filed under: Uncategorized — Tags: , — Chris Haas @ 1:37 pm

Just recieved this error:

Could not find any resources appropriate for the specified culture or the neutral culture.  Make sure "AjaxControlToolkit.Properties.Resources.NET4.resources" was correctly embedded or linked into assembly "AjaxControlToolkit" at compile time, or that all the satellite assemblies required are loadable and fully signed.

Sometimes .Net errors look a lot more worse than they really are. In this case it had nothing to do with missing DLLs or a bad web.config but rather I simply forgot to register the TookitScriptManager before my CalendarExtender.

<ajax:ToolkitScriptManager ID="ToolkitScriptManager1" runat="server" />

August 11, 2010

Sql OUTPUT Params, SqlDataReaders and “Object reference not set to an instance of an object”

Filed under: Uncategorized — Tags: , — Chris Haas @ 12:15 pm

When you call ExecuteReader() on a SqlCommand object that has OUTPUT parameters you won’t be able to access the OUTPUT parameters until the SqlDataReader has been closed.

            Private Shared Sub SomeMethod()
                Using Con As New SqlConnection("DSN...")
                    Con.Open()
                    Using Com As New SqlCommand("MyProc", Con)
                        Com.CommandType = System.Data.CommandType.StoredProcedure
                        Com.Parameters.Add(New SqlParameter("@YourOutputParam", System.Data.SqlDbType.Int) With {.Direction = Data.ParameterDirection.Output})
                        Using RDR = Com.ExecuteReader()
                            ''Using the line below will fail
                            'Dim X = Com.Parameters("@YourOutputParam").Value
                            Do While RDR.Read
                                'Do stuff with your recordset...
                            Loop
                        End Using
                        ''This will work because the Using block closes the SqlDataReader
                        'Dim X = Com.Parameters("@YourOutputParam").Value
                    End Using
                    Con.Close()
                End Using
            End Sub

November 2, 2009

Fixing “The remote server returned an error: (417) Expectation Failed”

Filed under: Uncategorized — Tags: — Chris Haas @ 3:46 pm

After failing at breakpointing and debugging request and response headers I almost snapped Wireshark to my connection to determine why I was getting an WebException of “The remote server returned an error: (417) Expectation Failed“. A little searching took me to this site with the solution. The simple solution is to set the static property System.Net.ServicePointManager.Expect100Continue to False before making your call.

Documentation of that property can be found here.

June 25, 2009

Creating a simple multi-threaded VB.Net application

Filed under: Uncategorized — Tags: — Chris Haas @ 11:41 am

I write lots of desktop tools that help me do my job more efficiently. For instance, I’m normalizing a 98 million row live table and do to the nature of the server, table locks, etc, I’ve decided to pull the data down in batches using a VB.Net app, analyze it and perform a batch update. The high-level code is simple: pull the data, group it and then execute some UPDATE SQL statements. But 98 million rows in a partially non-indexed table takes a looooong time, and maybe its my agency background, but I want some feedback as to what’s going on, and I want more than just a Trace/Console output. At first glance you might just add a progress bar and update it every time you do something, but unfortunately your code loop will block the UI thread from updating until complete (although you can throw some Application.DoEvents in there).

So enter multi-threading. If you think this is overkill then a) you obviously don’t know me and b) I’ve been doing this for years, it only takes a little bit more work to add it. The code below will show you how to create a multi-threaded app with a responsive UI. The code below is all done in Visual Basic 2008 Express Edition.

1) Create a new VB.Net Windows Forms Application.
2) Add a ProgressBar and a Button to the form, leave the names the default names
3) Resize the ProgressBar so you can see more of it
4) Double-click the Button to go into code-behind
5) Add a new class to the project called “Worker” and add the following code to it:


Option Explicit On
Option Strict On

Public Class Worker
    Private _curRun As Integer
    Private _maxRuns As Integer
    Public ReadOnly Property CurRun() As Integer
        Get
            Return Me._curRun
        End Get
    End Property
    Public ReadOnly Property MaxRuns() As Integer
        Get
            Return Me._maxRuns
        End Get
    End Property
    Public Sub New(ByVal maxRuns As Integer)
        Me._maxRuns = maxRuns
    End Sub
    Public Sub Start()
        For Me._curRun = 1 To Me._maxRuns
            System.Threading.Thread.Sleep(250)
        Next
    End Sub
End Class

This code will be doing our primary work. In this case its not actually doing anything except sleeping, you’ll want to replace the System.Threading.Thread.Sleep(250) with your own logic, obviously. We’ll instantiate this code in our primary UI thread later, telling it how many times we want it to loop. I always prefer to pass my required variables in the constructor and make them read-only properties so that I know that they’ll be set. One other thing that you’ll notice, usually in the For loop you’d do something like For I = 1 to Me._maxRuns, creating a local counter variable. Instead, our code creates a class-level variable and gives read-only permission to it so that other code (on another thread) can determine where we’re at. That’s it from the worker class’s point of view. The only bit of thread-related code in here is the Sleep method and that’s for example purposes only.

The next step is to add some logic to the main form’s code-behind to create our worker, create a thread to execute the worker and create a thread to monitor the worker and report back to the form and changes. Before I slap the code up let me explain how it will work. After clicking the button the form’s primary (and only, initially) thread will create an instance of the Worker class. It will then create a new Thread object and tell it that when we execute it it should call the Worker’s Start() method. This alone would give you a responsive UI but unfortunately the Worker thread can’t safely update the UI because they’re on different threads (and VS 2008 will actually throw errors if you try). You can get around this using events or delegates in the worker object but I prefer to make my worker object 100% completely thread un-aware, meaning that it doesn’t report back to anyone, it just does the work and if someone wants to inspect its progress, fine, but it doesn’t really care. So how do we do this? We introduce a third thread whose only job is to check the status of the worker object and tell the main thread to update the progress bar. The code is so simple that I don’t even bother wrapping it in a separate class, I just include a Monitor() method in the form’s code-behind. The code runs a loop until the worker thread signals that its done, sleeping for a bit between each loop so that we don’t flood the UI with update calls (there’s no sense in sending “set the progress bar to 1″ a million times a second). Threads automatically handle adjusting their ThreadState so we don’t have to worry about that.

One of the things that takes a little bit of getting used to is the concept that one thread cannot modify controls on another thread. So our Monitor() method can’t actually update the UI. Instead what it does is ask the UI thread to update the UI on behalf of the monitor thread. I know, sounds weird. To do this, our UpdateUI method first calls Me.InvokeRequired which basically asks the form to check if the code calling this method is on the same thread. If its on the same thread it just adjust the progress bar. If its not on the same thread it ask the form to call the same method with the same parameters on its behalf using the Invoke() method. Invoke takes two parameters although the second is optional. The first is a delegate which is basically a type-safe function pointer. If you were using Javascript this would be the same as passing the function’s name without parentheses or just using the function(){} syntax. The second is an array of objects that represent the parameters to pass to the method. The syntax may look confusing but that’s just VB’s way of declaring an array. I think they’re fixing that in VB10 but I’m not sure. We also use a similar concept when the main thread is done to re-enable the button. And that’s it.


Option Explicit On
Option Strict On

Imports System.Threading

Public Class Form1
    Private MonitorThread As Thread
    Private WorkerThread As Thread
    Private W As Worker
    Private Delegate Sub UpdateUIDelegate(ByVal curIndex As Integer)
    Private Delegate Sub WorkerDoneDelegate()

    Private Sub Button1_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Button1.Click
        Me.Button1.Enabled = False                      'Disable the button
        W = New Worker(50)                              'Create our Worker object
        Me.ProgressBar1.Maximum = W.MaxRuns             'Set the maximum value of the progress bar
        WorkerThread = New Thread(AddressOf W.Start)    'Create our Worker thread and tell it that when we start it it should call our Worker's Start() method
        MonitorThread = New Thread(AddressOf Monitor)   'Create our Monitor thread and tell it that when we start it it should call this class's Monitor() method
        WorkerThread.Start()                            'Start the worker thread
        MonitorThread.Start()                           'Start the monitor thread
    End Sub
    Private Sub Monitor()
        Do While WorkerThread.ThreadState <> ThreadState.Stopped    'Loop until the Worker thread (and thus the Worker object's Start() method) is done
            UpdateUI(W.CurRun)                                      'Update the progress bar with the current value
            Thread.Sleep(250)                                       'Sleep the monitor thread otherwise we'll be wasting CPU cycles updating the progress bar a million times a second
        Loop
        WorkerDone()                                                'If we're here, the worker object is done, call a method to do some cleanup
    End Sub
    Private Sub UpdateUI(ByVal curIndex As Integer)
        If Me.InvokeRequired Then                                                           'See if we need to cross threads
            Me.Invoke(New UpdateUIDelegate(AddressOf UpdateUI), New Object() {curIndex})    'If so, have the UI thread call this method for us
        Else
            Me.ProgressBar1.Value = curIndex                                                'Otherwise just update the progress bar
        End If
    End Sub
    Private Sub WorkerDone()
        If Me.InvokeRequired Then                                                           'See if we need to cross threads
            Me.Invoke(New WorkerDoneDelegate(AddressOf WorkerDone))                         'If so, have the UI thread call this method for us
        Else
            Me.Button1.Enabled = True                                                       'Otherwise just update the button
        End If
    End Sub
End Class

One last thing. Debugging a multi-threaded program can be a PITA so one of the things that I do is include a switch to turn it on or off as needed. I prefer to do this with a Compiler Directive by adding #Const MULTI_THREADED = True to the top of my code-behind. Then there’s just a simple change to the button’s click event:


    Private Sub Button1_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Button1.Click
        Me.Button1.Enabled = False                      'Disable the button
        W = New Worker(50)                              'Create our Worker object
#If MULTI_THREADED Then
        Me.ProgressBar1.Maximum = W.MaxRuns             'Set the maximum value of the progress bar
        WorkerThread = New Thread(AddressOf W.Start)    'Create our Worker thread and tell it that when we start it it should call our Worker's Start() method
        MonitorThread = New Thread(AddressOf Monitor)   'Create our Monitor thread and tell it that when we start it it should call this class's Monitor() method
        WorkerThread.Start()                            'Start the worker thread
        MonitorThread.Start()                           'Start the monitor thread
#Else
        W.Start()
        Me.Button1.Enabled = True
#End If
    End Sub

When MULTI_THREADED is False we’ll invoke the worker, blocking the UI thread from ever updating, do the work and reset the button. This can save you a lot of time when debugging. This is also why I like to keep my worker object thread un-aware.

Fixing “query processor ran out of stack space”

Filed under: Uncategorized — Tags: , , — Chris Haas @ 9:07 am

I’ve been normalizing a table with 98 million rows for a couple of days now and with a recent change I’ve been able to run my batches in groups of 100,000 instead of just 10,000. The program doing the normalization pulls down 100,000 records at a time, analyzes and groups them and performs an UPDATE for each item in the group, passing the primary keys back to the server in an IN () clause. Everything was fine for a while but with the upgrade to 100,000 and with some groupings sending thousands of keys in the IN clause I started to get the following error:

Internal Query Processor Error: The Query Processor Ran Out Of Stack Space During Query Optimization.

Microsoft says to insert the keys into a temp table and join with that but that doesn’t really work for me. Instead I just decided to break up the UPDATE query into ones with more manageable IN clauses.

Originally I had something like:


            Dim mySQL = String.Format("UPDATE Tbl SET Col2=xyz WHERE Col1 IN ({0})", Join(TheList.ToArray(), ","))
            Using Com As New SqlCommand(mySQL, con)
                Com.CommandType = CommandType.Text
                Com.ExecuteNonQuery()
            End Using

The new code breaks things into more manageable sizes:


        Dim AmmountToProcess = 1000
        Do While True
            Dim Y = TheList.Take(AmmountToProcess)
            Dim mySQL = String.Format("UPDATE Tbl SET Col2=xyz WHERE Col1 IN ({0})", Join(Y.ToArray(), ","))
            Using Com As New SqlCommand(mySQL, con)
                Com.CommandType = CommandType.Text
                Com.ExecuteNonQuery()
            End Using
            TheList.RemoveRange(0, Math.Min(TheList.Count, AmmountToProcess))
            If TheList.Count = 0 Then Exit Do
        Loop

The new code uses the wonderful Take extension method to grab 1,000 items. Take luckily doesn’t error if you ask for more than available. The smaller UPDATE query runs and then we remove the first 1,000 items (or however many are left) from the list. Repeat until the list is empty.

May 15, 2009

ASP.Net PNG and the dreaded “a generic error occurred in gdi+”

Filed under: Uncategorized — Tags: — Chris Haas @ 9:27 pm

If you ever make a PNG on the fly using ASP.Net and try to write it directly to the Response.OutputStream you’ll get the exception “a generic error occurred in gdi+“. If you change the output type to JPG or GIF it runs just fine, just PNG breaks.

Now usually .Net exceptions are pretty easy to trace (at least for me) just by name but if you try searching for a generic error guess what you get? Generic results. The problem is that the PNG encoder needs a “seekable stream” when outputing the image and the Response.OutputStream is not seekable. Luckily the fix is easy, just write it to a MemoryStream (which is seekable) and then BinaryWrite the MemoryStream.

Assume that BI is a System.Drawing.Image. Instead of:

BI.Save(Response.OutputStream, System.Drawing.Imaging.ImageFormat.Png)

you need to use:

Using MS As New MemoryStream()
    BI.Save(MS, System.Drawing.Imaging.ImageFormat.Png)
    Response.BinaryWrite(MS.GetBuffer())
End Using

April 22, 2009

True ASP.Net (and IIS) 404 errors

Filed under: Uncategorized — Tags: , — Chris Haas @ 9:26 pm

UPDATE: I should found out that in ASP.Net 3.5 the <customErrors> section in web.config now supports the attribute redirectMode which default to ResponseRedirect but can be changed to ResponseRewrite. This will force the proper HTTP status codes to come through for errors.

So I launched a new site porting it from static .html files to dynamic .aspx. I then created an entry for 404′s in web.config and also told IIS to send 404′s to the same page for everything that ASP.Net didn’t handle (images, html files, etc). So simple, right? Unfortunately both ASP.Net and IIS don’t appear to actually send a 404 status code for 404′s if you have a 404 handler specified. IIS actually sends a 200 OK and performs the equivalent of a Server.Transfer. ASP.Net actually sends a 302 Found which is a client-side redirect.

Now, you may not care about this, but if you’re trying to purge search engines of old content you should.

The fix, at first glance, is simple. In Page_Load on the 404 handler page just send the proper status code using Response.StatusCode = 404. This actually fixes the problem with IIS but we’ve still got an ASP.Net problem. ASP.Net will actually still send the 302 redirecting people to the 404 page and then send the 404 code. So if page Blah.aspx doesn’t exist you’ll get a 302 code saying to go to PageNotFound.aspx. When the browser requests PageNotFound.aspx they finally get the 404 response. Now maybe that’s good enough for you, but not for me. When I 404, I want to know right away, not after a redirect.

I found the solution here and have reproduced the code below for easy copy-and-paste. They don’t explicitly mention it in the article but you MUST set the <customErrors> to Off or at least RemoteOnly. I know that sounds weird but the code below still actually processes the <customErrors>, it just does it instead of letting ASP.Net handle it for you.

The article talks about creating a DLL but you can just as easily drop it into App_Code as a file named HttpErrorModule. Then just add <add name="HttpErrorModule" type="HttpErrorModule" /> to your <httpModules>.

One other thing to note, this module actually overrides all errors and sends their proper codes in the response stream. This shouldn’t cause any problems but you should be aware of it when debugging. You could very easily add some logic to check for 404′s only, too.

'Code from: http://www.colincochrane.com/post/2008/01/ASP-NET-Custom-Errors-Preventing-302-Redirects-To-Custom-Error-Pages.aspx
Option Strict On
Option Explicit On

Imports System.Web
Imports System.Web.Configuration

Public Class HttpErrorModule
    Implements IHttpModule

    Public Sub Dispose() Implements System.Web.IHttpModule.Dispose
        'Nothing to dispose.
    End Sub
    Public Sub Init(ByVal context As System.Web.HttpApplication) Implements System.Web.IHttpModule.Init
        AddHandler context.Error, New EventHandler(AddressOf Context_Error)
    End Sub

    Private Sub Context_Error(ByVal sender As Object, ByVal e As EventArgs)
        Dim context As HttpContext = CType(sender, HttpApplication).Context
        If (context.Error.GetType Is GetType(HttpException)) Then
            ' Get the Web application configuration.
            Dim configuration As System.Configuration.Configuration = WebConfigurationManager.OpenWebConfiguration("~/web.config")

            ' Get the section.
            Dim customErrorsSection As CustomErrorsSection = CType(configuration.GetSection("system.web/customErrors"), CustomErrorsSection)

            ' Get the collection
            Dim customErrorsCollection As CustomErrorCollection = customErrorsSection.Errors
            Dim statusCode As Integer = CType(context.Error, HttpException).GetHttpCode

            'Clears existing response headers and sets the desired ones.
            context.Response.ClearHeaders()
            context.Response.StatusCode = statusCode
            If (customErrorsCollection.Item(statusCode.ToString) IsNot Nothing) Then
                context.Server.Transfer(customErrorsCollection.Get(statusCode.ToString).Redirect)
            Else
                context.Response.Flush()
            End If
        End If
    End Sub
End Class
Older Posts »

Powered by WordPress