Advanced Application and Server Troubleshooting Techniques Edgewebhosting net

Personal Background Vlad A. Friedman n CEO, Edgewebhosting. net n n #1 Rated CF Managed Hosting Provider Hosting since 1998 n Operating one of the Most Redundant Datacenters in the world n Hundreds of Servers running over 20, 000 websites n

Troubleshooting Overview n Different Types of Application Servers n Common Problems and Fixes n Troubleshooting Tools & Techniques n How does Edgeweb locate and eradicate problems and how does that relate to your environment?

Types of Application Servers n Cold Fusion / Blue Dragon n n Web application server Dependent on n n IIS/Apache n n Stores and Retrieves Data Mail Servers n n Receives requests from clients Routes requests to application server Returns results back to clients SQL Server/My. SQL Server/MS Access n n Code IIS SQL Mail Servers Sends messages and notifications to users All of the above subsystems must run efficiently for an application to function and fulfill its purpose.

Application Dependency Pyramid It is important to understand The role of each layer in the application pyramid. If any lower layer fails, none of the layers above it can function. Cold Fusion Blue Dragon IIS, JRUN SQL Server, Mail Servers Operating Systems Network Infrastructure Hardware Infrastructure

Common Problems n Completely Unresponsive servers. n Crashing Applications (Nothing is working) n n Errors Hanging Applications (in CF and BD) Web Server unresponsive Slow application response time n n Bad Code Custom Tags Bad Queries Misconfiguration

Completely Unresponsive Servers (Hardware) n Check the Hardware n 1) Try to ping the server (if ping is allowed in your environment). This is the best way to tell if it is still alive without getting up from your desk. n 2) Get physical access on the server. If it is unresponsive or has a BSOD, consider a reboot. 3) Following reboot, check Windows Event Log for errors Physical Locks and BSOD (Blue Screen of Death) Lockups can be caused by a number of physical components n BAD CPU Hardware Infrastructure n BAD Memory n BAD Power Supply n BAD Hard Disks, especially RAID n n n Ensure hardware is working properly prior to continuing.

Completely Unresponsive Servers (Network) n Check the Network n Ensure all network connections are in place and working n If ping is allowed in your environment n n Ping the server from your workstation Ping your workstation from the server i. e. n ping servername Network Infrastructure n ping workstation If you have access to the server, it appears to be operational, but you cannot ping any machines from it, check your network settings and connections. Run a trace route with the following command from your workstation: n tracert servername You may have a network problem outside of your server and workstation environment preventing you from communicating. Trace route shows the path data travels across the internet between two locations.

Free Internet/Network Troubleshooting Tools n A number of free tools are available online to allow you to check your server from multiple worldwide locations to troubleshoot external network problems: n http: //visualroute. visualware. com/ n http: //www. alertra. com n http: //www. checkdns. org

http: //visualroute. visualware. com/ Online visual trace route

http: //www. alertra. com Free Online site check from 6 worldwide locations

Troubleshooting Operating Systems n Common Problems n Virus Infected n Use free online virus scanner n n Now accounts for over 20% of all server infections Sys. Internals Root Kit Revealer n n n http: //housecall. trendmicro. com/ http: //www. pandasoftware. com/activescan Root Kits n n Operating Systems http: //www. sysinternals. com/utilities/rootkitrevealer. html Corrupted Registry Corrupted files in OS Microsoft “Security” Patches Always check the Event. Log for problems. Always make sure your server has the latest patches applied

Troubleshooting IIS Internet Information Server Receives requests from multiple clients simultaneously. n Dispatches requests to application servers n Issues data returned by application server back to the correct client. n n Common problems include: OS Permissions on file system n Misconfiguration n Corrupt Metabase n IIS, JRUN SQL Server, Mail Servers

Installing and using IISTracer n Available from http: //www. iismonitor. net n IISTracer is a real-time internet web site monitoring tool for Microsoft IIS (all versions). The software lets you show what is happening on IIS server right now. n You can see or log state of each request, incoming outgoing bytes, http headers and several request running times. n IISTracer shows a state of running scripts (. asp, . aspx, asp. net, . cfm, . php, . cgi, . . . ), applications (. dll, . exe, . . ) and a big downloads (. mp 3, . zip, . . . ). n Free to try for 60 days, $99 to Buy

IISTracer Screenshot

Troubleshooting IIS with Netstat n n Netstat allows you to view connections in real time as well as services that are listening on your server. IIS Runs on ports: n n n Run n TCP TCP 80 for HTTP 443 for HTTPS 21 for FTP 25 for SMTP netstat –an Active Connections Proto TCP TCP TCP TCP TCP Local Address 0. 0: 80 0. 0: 135 0. 0: 443 0. 0: 445 0. 0: 1039 0. 0: 1160 0. 0: 2368 0. 0: 2948 0. 0: 3389 0. 0: 4678 127. 0. 0. 1: 1051 127. 0. 0. 1: 4664 127. 0. 0. 1: 5679 208. 39. 184. 203: 139 208. 39. 184. 203: 80 Foreign Address 0. 0: 0 0. 0: 0 0. 0: 0 208. 39. 184. 217: 1445 198. 65. 119. 20: 3380 69. 63. 130. 5: 4139 State LISTENING LISTENING LISTENING LISTENING ESTABLISHED

Troubleshooting with Telnet n Telnet allow you to create a low level connection to your server to see if it is evening listening for requests. Run telnet servername 80 n If you get a blank screen, the server is listening. Try typing (case is important) GET /index. cfm n n n This will return the actual data that your browser would normally receive. If you do not get a response, there is something wrong with IIS or CF. Try getting an html page from the site. i. e. GET /test. htm n If it returns valid results, then there is a problem with CF, but IIS is working properly.

Other helpful tools n Free tools available from Sysinternals. com n File. Mon n Shows real-time file activity, permissions and file access problems. n Reg. Mon n Shows n Pskill n Allows n real-time registry activity you to kill running processes Tcp. View n View TCP/IP connections in real time.

Troubleshooting SQL Server n n If you seem to be getting slow response from only your dynamic pages, and all of the static pages are working fine, check your database. Use Microsoft SQL Profiler to connect to your database to measure performance. n n n SQL Server Profiler will show you every query running on the server in real time and amount of resources used on the server. Generally, most normal optimized queries have a CPU of under 20 and READs of under 100. A high number of reads means that you are missing indexes and the server is having to physically scan records in your tables.

SQL Profiler Click File->New->Trace n Login to the database server n Use the default settings n

Query Analyzer n Use query analyzer to see the detailed performance usage breakdown on each query n n n Run query analyzer Connect to your database Click Query-> n n Show Execution Plan Show Server Trace Show Client Statistics Then cut and paste your query from Profiler into the query window, and hit the run button.

Query Analyzer Results

Troubleshooting Mail Servers n Check your Cold Fusion mail spool folder to ensure messages are being removed from the queue n n If messages are not moving, try moving them to a temporary location, and restarting CF. If mail starts flowing, check the messages that you moved for corruption. Check the undeliverable folder to make sure that messages are not getting rejected. n n n CF 5 – c: cfusionmailspool CFMX – c: CFusion. MXmailspool CF 5 - C: CFusionmailUn. Delivr CFMX - C: CFusion. MXmailUn. Delivr Enable mail logging in CF administrator and check the CF mail logs to ensure messages are being processed.

Troubleshooting Mail Servers n Check the mail server n n n Mail sure you are using the correct mail server Review the mail logs and event logs and look for errors. Check that Mail server is able to resolve DNS using nslookup on the mail server itself nslookup Default Server: mail. edgewebhosting. net Address: 69. 63. 128. 71 > set type=mx > ewh. net Server: mail. edgewebhosting. net Address: 69. 63. 128. 71 Non-authoritative answer: ewh. net MX preference = 10, mail exchanger = relay 1. edgewebhosting. net ewh. net nameserver = ns 2. edgewebhosting. net ewh. net nameserver = ns 1. edgewebhosting. net relay 1. edgewebhosting. net internet address = 69. 63. 128. 201 ns 1. edgewebhosting. net internet address = 69. 63. 128. 230 ns 2. edgewebhosting. net internet address = 69. 63. 128. 231 n Try telnet to the server on port 25 on your Cold Fusion server n telnet mailserver 25 n This should return a 220 response: 220 ewhserver 18. edgewebhosting. net (IMail 8. 20 8743 -27) NT-ESMTP Server

Cold Fusion and Blue Dragon n Top of the Pyramid Cold Fusion Blue Dragon

Crashing Applications n Programming Errors n If your code hasn’t changed, check what has changed n n Operating System, patches and upgrades Application server patches and upgrades Try restarting Cold Fusion and web services or rebooting the server to see if the error condition clears itself. If your code has changed n n Make sure you have an error handler on your site. Check the Cold Fusion Log Files for Errors n n In Cold Fusion 5 n C: CFusionLog In Cold Fusion MX n C: CFusion. MXLogs n C: CFusion. MXruntimelogs n Check the Windows Event. Logs for Errors Enable Cold Fusion Debugging Programming Errors are typically the easiest type of problem to identify

Hanging Applications and Web Server Unresponsiveness n When you hit your site or application and the browser just spins n Try connecting to port 80 on the server n n telnet servername 80 GET /pagename. cfm n You should receive the results from CF. If not, work your way back down the pyramid. n Enable logging of long running requests in CF Administrator n Use a 3 rd party application to troubleshoot n See. Fusion from See. Fusion. com (2 hour trial) n Provides very detailed debugging information. n Allows you to see running pages and queries together in debug output. n Has a running debug mode to see all debugging information from every page in real time.

Debugging CF and BD with See. Fusion. com n View Cold Fusion and Db Requests in Real Time

Installation Instructions n Online Instructions at: n http: //www. seefusion. com/index. cfm? do=c_guide. install n Download the seefusion. jar file from http: //www. seefusion. com/seefusion. jar and place the file in c: cfusionmxwwwrootweb-inflib n download the sample properties file from http: //www. seefusion. com/seefusion. properties and place the file in c: cfusionmxwwwrootweb-infclasses n n n edit web. xml, at first servlet definition <filter> <filter-name>See. Fusion</filter-name> <filter-class>com. seefusion. See. Fusion</filter-class> </filter> <filter-mapping> <filter-name>See. Fusion</filter-name> <url-pattern>/*</url-pattern> </filter-mapping> Restart CFMX Open your browser to http: //localhost: 8999

Walkthrough of Actual Installation of See. Fusion

The End and Thank you n Questions? n Shameless Plug: If you have more questions, or would like to know how Edgewebhosting. net’s high capacity managed hosting solutions can benefit your organization, please contact us at 1 -866 -EDGEWEB.