NetUtils (Apache Hadoop Common 3.5.0 API)
Description: declaration: package: org.apache.hadoop.net, class: NetUtils

Package org.apache.hadoop.net

# Class NetUtils

java.lang.Object
org.apache.hadoop.net.NetUtils

@LimitedPrivate({"HDFS","MapReduce"}) @Unstable public class NetUtils extends Object

- ## Field Summary

  Fields
  Modifier and Type
  Field
  Description
  `static final String`
  `HADOOP_WIKI`
  Base URL of the Hadoop Wiki: "http://wiki.apache.org/hadoop/"
  `static final String`
  `UNKNOWN_HOST`
  text included in wrapped exceptions if the host is null: "(unknown)"

- ## Constructor Summary

  Constructors
  Constructor
  Description
  `NetUtils()`

- ## Method Summary

  Modifier and Type
  Method
  Description
  `static IOException`
  `addNodeNameToIOException(IOException ioe, String nodeName)`
  Return an @`IOException` of the same type as the input exception but with a modified exception message that includes the node name.
  `static void`
  `addStaticResolution(String host, String resolvedName)`
  Adds a static resolution for host.
  `static InetAddress`
  `bindToLocalAddress(InetAddress localAddr, boolean bindWildCardAddress)`
  Return an @`InetAddress` to bind to.
  `static void`
  `connect(Socket socket, SocketAddress address, int timeout)`
  This is a drop-in replacement for `Socket.connect(SocketAddress, int)`.
  `static void`
  `connect(Socket socket, SocketAddress endpoint, SocketAddress localAddr, int timeout)`
  Like `connect(Socket, SocketAddress, int)` but also takes a local address and port to bind the socket to.
  `static InetSocketAddress`
  `createSocketAddr(String target)`
  Util method to build socket addr from either.
  `static InetSocketAddress`
  `createSocketAddr(String target, int defaultPort)`
  Util method to build socket addr from either.
  `static InetSocketAddress`
  `createSocketAddr(String target, int defaultPort, String configName)`
  Create an InetSocketAddress from the given target string and default port.
  `static InetSocketAddress`
  `createSocketAddr(String target, int defaultPort, String configName, boolean useCacheIfPresent)`
  Create an InetSocketAddress from the given target string and default port.
  `static InetSocketAddress`
  `createSocketAddr(String target, int defaultPort, String configName, boolean useCacheIfPresent, boolean isResolved)`
  `static InetSocketAddress`
  `createSocketAddrForHost(String host, int port)`
  Create a socket address with the given host and port.
  `static InetSocketAddress`
  `createSocketAddrUnresolved(String target)`
  `static List<String[]>`
  `getAllStaticResolutions()`
  This is used to get all the resolutions that were added using `addStaticResolution(String, String)`.
  `static URI`
  `getCanonicalUri(URI uri, int defaultPort)`
  Resolve the uri's hostname and add the default port if not in the uri
  `static InetSocketAddress`
  `getConnectAddress(InetSocketAddress addr)`
  Returns an InetSocketAddress that a client can use to connect to the given listening address.
  `static InetSocketAddress`
  `getConnectAddress(Server server)`
  Returns InetSocketAddress that a client can use to connect to the server.
  `static SocketFactory`
  `getDefaultSocketFactory(Configuration conf)`
  Get the default socket factory as specified by the configuration parameter `hadoop.rpc.socket.factory.default`
  `static int`
  `getFreeSocketPort()`
  Return a free port number.
  `static Set<Integer>`
  `getFreeSocketPorts(int numOfPorts)`
  Return free ports.
  `static String`
  `getHostname()`
  Return hostname without throwing exception.
  `static String`
  `getHostNameOfIP(String ipPort)`
  Attempt to obtain the host name of the given string which contains an IP address and an optional port.
  `static String`
  `getHostPortString(InetSocketAddress addr)`
  Compose a "host:port" string from the address.
  `static SocketInputWrapper`
  `getInputStream(Socket socket)`
  Same as `getInputStream(socket, socket.getSoTimeout()).`
  `static SocketInputWrapper`
  `getInputStream(Socket socket, long timeout)`
  Return a `SocketInputWrapper` for the socket and set the given timeout.
  `static List<InetAddress>`
  `getIPs(String subnet, boolean returnSubinterfaces)`
  Return an InetAddress for each interface that matches the given subnet specified using CIDR notation.
  `static String`
  `getLocalHostname()`
  Return hostname without throwing exception.
  `static InetAddress`
  `getLocalInetAddress(String host)`
  Checks if `host` is a local host name and return `InetAddress` corresponding to that address.
  `static OutputStream`
  `getOutputStream(Socket socket)`
  Same as getOutputStream(socket, 0).
  `static OutputStream`
  `getOutputStream(Socket socket, long timeout)`
  Returns OutputStream for the socket.
  `static int`
  `getPortFromHostPortString(String addr)`
  Get port as integer from host port string like host:port.
  `static SocketFactory`
  `getSocketFactory(Configuration conf, Class<?> clazz)`
  Get the socket factory for the given class according to its configuration parameter `hadoop.rpc.socket.factory.class.<ClassName>`.
  `static SocketFactory`
  `getSocketFactoryFromProperty(Configuration conf, String propValue)`
  Get the socket factory corresponding to the given proxy URI.
  `static String`
  `getStaticResolution(String host)`
  Retrieves the resolved name for the passed host.
  `static boolean`
  `isLocalAddress(InetAddress addr)`
  Given an InetAddress, checks to see if the address is a local address, by comparing the address with all the interfaces on the node.
  `static boolean`
  `isValidSubnet(String subnet)`
  isValidSubnet.
  `static String`
  `normalizeHostName(String name)`
  Given a string representation of a host, return its ip address in textual presentation.
  `static List<String>`
  `normalizeHostNames(Collection<String> names)`
  Given a collection of string representation of hosts, return a list of corresponding IP addresses in the textual representation.
  `static String`
  `normalizeIP2HostName(String ipPort)`
  Attempt to normalize the given string to "host:port" if it like "ip:port".
  `static void`
  `verifyHostnames(String[] names)`
  Performs a sanity check on the list of hostnames/IPs to verify they at least appear to be valid.
  `static IOException`
  `wrapException(String destHost, int destPort, String localHost, int localPort, IOException exception)`
  Take an IOException , the local host port and remote host port details and return an IOException with the input exception as the cause and also include the host details.

  ### Methods inherited from class java.lang.Object

  `clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- ## Field Details

  - ### UNKNOWN_HOST

    public static final String UNKNOWN_HOST
    text included in wrapped exceptions if the host is null: "(unknown)"
    See Also:
      - Constant Field Values

  - ### HADOOP_WIKI

    public static final String HADOOP_WIKI
    Base URL of the Hadoop Wiki: "http://wiki.apache.org/hadoop/"
    See Also:
      - Constant Field Values

- ## Constructor Details

  - ### NetUtils

    public NetUtils()

- ## Method Details

  - ### getSocketFactory

    public static SocketFactory getSocketFactory(Configuration conf, Class<?> clazz)
    Get the socket factory for the given class according to its configuration parameter `hadoop.rpc.socket.factory.class.<ClassName>`. When no such parameter exists then fall back on the default socket factory as configured by `hadoop.rpc.socket.factory.class.default`. If this default socket factory is not configured, then fall back on the JVM default socket factory.
    Parameters:
      `conf` - the configuration
      `clazz` - the class (usually a `VersionedProtocol`)
    Returns:
      a socket factory

  - ### getDefaultSocketFactory

    public static SocketFactory getDefaultSocketFactory(Configuration conf)
    Get the default socket factory as specified by the configuration parameter `hadoop.rpc.socket.factory.default`
    Parameters:
      `conf` - the configuration
    Returns:
      the default socket factory as specified in the configuration or the JVM default socket factory if the configuration does not contain a default socket factory property.

  - ### getSocketFactoryFromProperty

    public static SocketFactory getSocketFactoryFromProperty(Configuration conf, String propValue)
    Get the socket factory corresponding to the given proxy URI. If the given proxy URI corresponds to an absence of configuration parameter, returns null. If the URI is malformed raises an exception.
    Parameters:
      `conf` - configuration.
      `propValue` - the property which is the class name of the SocketFactory to instantiate; assumed non null and non empty.
    Returns:
      a socket factory as defined in the property value.

  - ### createSocketAddr

    public static InetSocketAddress createSocketAddr(String target)
    Util method to build socket addr from either. <host>:<port> <fs>://<host>:<port>/<path>
    Parameters:
      `target` - target.
    Returns:
      socket addr.

  - ### createSocketAddrUnresolved

    public static InetSocketAddress createSocketAddrUnresolved(String target)

  - ### createSocketAddr

    public static InetSocketAddress createSocketAddr(String target, int defaultPort)
    Util method to build socket addr from either. <host> <host>:<port> <fs>://<host>:<port>/<path>
    Parameters:
      `target` - target.
      `defaultPort` - default port.
    Returns:
      socket addr.

  - ### createSocketAddr

    public static InetSocketAddress createSocketAddr(String target, int defaultPort, String configName)
    Create an InetSocketAddress from the given target string and default port. If the string cannot be parsed correctly, the `configName` parameter is used as part of the exception message, allowing the user to better diagnose the misconfiguration.
    Parameters:
      `target` - a string of either "host" or "host:port"
      `defaultPort` - the default port if `target` does not include a port number
      `configName` - the name of the configuration from which `target` was loaded. This is used in the exception message in the case that parsing fails.
    Returns:
      socket addr.

  - ### createSocketAddr

    public static InetSocketAddress createSocketAddr(String target, int defaultPort, String configName, boolean useCacheIfPresent)
    Create an InetSocketAddress from the given target string and default port. If the string cannot be parsed correctly, the `configName` parameter is used as part of the exception message, allowing the user to better diagnose the misconfiguration.
    Parameters:
      `target` - a string of either "host" or "host:port"
      `defaultPort` - the default port if `target` does not include a port number
      `configName` - the name of the configuration from which `target` was loaded. This is used in the exception message in the case that parsing fails.
      `useCacheIfPresent` - Whether use cache when create URI
    Returns:
      socket addr

  - ### createSocketAddr

    public static InetSocketAddress createSocketAddr(String target, int defaultPort, String configName, boolean useCacheIfPresent, boolean isResolved)

  - ### createSocketAddrForHost

    public static InetSocketAddress createSocketAddrForHost(String host, int port)
    Create a socket address with the given host and port. The hostname might be replaced with another host that was set via `addStaticResolution(String, String)`. The value of hadoop.security.token.service.use_ip will determine whether the standard java host resolver is used, or if the fully qualified resolver is used.
    Parameters:
      `host` - the hostname or IP use to instantiate the object
      `port` - the port number
    Returns:
      InetSocketAddress

  - ### getCanonicalUri

    public static URI getCanonicalUri(URI uri, int defaultPort)
    Resolve the uri's hostname and add the default port if not in the uri
    Parameters:
      `uri` - to resolve
      `defaultPort` - if none is given
    Returns:
      URI

  - ### addStaticResolution

    public static void addStaticResolution(String host, String resolvedName)
    Adds a static resolution for host. This can be used for setting up hostnames with names that are fake to point to a well known host. For e.g. in some testcases we require to have daemons with different hostnames running on the same machine. In order to create connections to these daemons, one can set up mappings from those hostnames to "localhost". `getStaticResolution(String)` can be used to query for the actual hostname.
    Parameters:
      `host` - the hostname or IP use to instantiate the object.
      `resolvedName` - resolved name.

  - ### getStaticResolution

    public static String getStaticResolution(String host)
    Retrieves the resolved name for the passed host. The resolved name must have been set earlier using `addStaticResolution(String, String)`
    Parameters:
      `host` - the hostname or IP use to instantiate the object.
    Returns:
      the resolution

  - ### getAllStaticResolutions

    public static List<String[]> getAllStaticResolutions()
    This is used to get all the resolutions that were added using `addStaticResolution(String, String)`. The return value is a List each element of which contains an array of String of the form String[0]=hostname, String[1]=resolved-hostname
    Returns:
      the list of resolutions

  - ### getConnectAddress

    public static InetSocketAddress getConnectAddress(Server server)
    Returns InetSocketAddress that a client can use to connect to the server. Server.getListenerAddress() is not correct when the server binds to "0.0.0.0". This returns "hostname:port" of the server, or "127.0.0.1:port" when the getListenerAddress() returns "0.0.0.0:port".
    Parameters:
      `server` - server.
    Returns:
      socket address that a client can use to connect to the server.

  - ### getConnectAddress

    public static InetSocketAddress getConnectAddress(InetSocketAddress addr)
    Returns an InetSocketAddress that a client can use to connect to the given listening address.
    Parameters:
      `addr` - of a listener
    Returns:
      socket address that a client can use to connect to the server.

  - ### getInputStream

    public static SocketInputWrapper getInputStream(Socket socket) throws IOException
    Same as `getInputStream(socket, socket.getSoTimeout()).`
    Parameters:
      `socket` - socket.
    Returns:
      SocketInputWrapper for reading from the socket.
    Throws:
      `IOException` - raised on errors performing I/O.
    See Also:
      - `getInputStream(Socket, long)`

  - ### getInputStream

    public static SocketInputWrapper getInputStream(Socket socket, long timeout) throws IOException
    Return a `SocketInputWrapper` for the socket and set the given timeout. If the socket does not have an associated channel, then its socket timeout will be set to the specified value. Otherwise, a `SocketInputStream` will be created which reads with the configured timeout. Any socket created using socket factories returned by `NetUtils()`, must use this interface instead of `Socket.getInputStream()`. In general, this should be called only once on each socket: see the note in `SocketInputWrapper.setTimeout(long)` for more information.
    Parameters:
      `socket` - socket.
      `timeout` - timeout in milliseconds. zero for waiting as long as necessary.
    Returns:
      SocketInputWrapper for reading from the socket.
    Throws:
      `IOException` - raised on errors performing I/O.
    See Also:
      - `Socket.getChannel()`

  - ### getOutputStream

    public static OutputStream getOutputStream(Socket socket) throws IOException
    Same as getOutputStream(socket, 0). Timeout of zero implies write will wait until data is available.
    From documentation for `getOutputStream(Socket, long)` :
    Returns OutputStream for the socket. If the socket has an associated SocketChannel then it returns a `SocketOutputStream` with the given timeout. If the socket does not have a channel, `Socket.getOutputStream()` is returned. In the later case, the timeout argument is ignored and the write will wait until data is available.
    Any socket created using socket factories returned by `NetUtils`, must use this interface instead of `Socket.getOutputStream()`.
    Parameters:
      `socket` - socket.
    Returns:
      OutputStream for writing to the socket.
    Throws:
      `IOException` - raised on errors performing I/O.
    See Also:
      - `getOutputStream(Socket, long)`

  - ### getOutputStream

    public static OutputStream getOutputStream(Socket socket, long timeout) throws IOException
    Returns OutputStream for the socket. If the socket has an associated SocketChannel then it returns a `SocketOutputStream` with the given timeout. If the socket does not have a channel, `Socket.getOutputStream()` is returned. In the later case, the timeout argument is ignored and the write will wait until data is available.
    Any socket created using socket factories returned by `NetUtils`, must use this interface instead of `Socket.getOutputStream()`.
    Parameters:
      `socket` - socket.
      `timeout` - timeout in milliseconds. This may not always apply. zero for waiting as long as necessary.
    Returns:
      OutputStream for writing to the socket.
    Throws:
      `IOException` - raised on errors performing I/O.
    See Also:
      - `Socket.getChannel()`

  - ### connect

    public static void connect(Socket socket, SocketAddress address, int timeout) throws IOException
    This is a drop-in replacement for `Socket.connect(SocketAddress, int)`. In the case of normal sockets that don't have associated channels, this just invokes `socket.connect(endpoint, timeout)`. If `socket.getChannel()` returns a non-null channel, connect is implemented using Hadoop's selectors. This is done mainly to avoid Sun's connect implementation from creating thread-local selectors, since Hadoop does not have control on when these are closed and could end up taking all the available file descriptors.
    Parameters:
      `socket` - socket.
      `address` - the remote address
      `timeout` - timeout in milliseconds
    Throws:
      `IOException` - raised on errors performing I/O.
    See Also:
      - `Socket.connect(java.net.SocketAddress, int)`

  - ### connect

    public static void connect(Socket socket, SocketAddress endpoint, SocketAddress localAddr, int timeout) throws IOException
    Like `connect(Socket, SocketAddress, int)` but also takes a local address and port to bind the socket to.
    Parameters:
      `socket` - socket.
      `endpoint` - the remote address
      `localAddr` - the local address to bind the socket to
      `timeout` - timeout in milliseconds
    Throws:
      `IOException` - raised on errors performing I/O.

  - ### normalizeHostName

    public static String normalizeHostName(String name)
    Given a string representation of a host, return its ip address in textual presentation.
    Parameters:
      `name` - a string representation of a host: either a textual representation its IP address or its host name
    Returns:
      its IP address in the string format

  - ### normalizeHostNames

    public static List<String> normalizeHostNames(Collection<String> names)
    Given a collection of string representation of hosts, return a list of corresponding IP addresses in the textual representation.
    Parameters:
      `names` - a collection of string representations of hosts
    Returns:
      a list of corresponding IP addresses in the string format
    See Also:
      - `normalizeHostName(String)`

  - ### verifyHostnames

    public static void verifyHostnames(String[] names) throws UnknownHostException
    Performs a sanity check on the list of hostnames/IPs to verify they at least appear to be valid.
    Parameters:
      `names` - - List of hostnames/IPs
    Throws:
      `UnknownHostException` - Unknown Host Exception.

  - ### getHostNameOfIP

    public static String getHostNameOfIP(String ipPort)
    Attempt to obtain the host name of the given string which contains an IP address and an optional port.
    Parameters:
      `ipPort` - string of form ip[:port]
    Returns:
      Host name or null if the name can not be determined

  - ### normalizeIP2HostName

    public static String normalizeIP2HostName(String ipPort)
    Attempt to normalize the given string to "host:port" if it like "ip:port".
    Parameters:
      `ipPort` - maybe lik ip:port or host:port.
    Returns:
      host:port

  - ### getLocalHostname

    public static String getLocalHostname()
    Return hostname without throwing exception. The returned hostname String format is "hostname".
    Returns:
      hostname

  - ### getHostname

    public static String getHostname()
    Return hostname without throwing exception. The returned hostname String format is "hostname/ip address".
    Returns:
      hostname

  - ### getHostPortString

    public static String getHostPortString(InetSocketAddress addr)
    Compose a "host:port" string from the address.
    Parameters:
      `addr` - address.
    Returns:
      hort port string.

  - ### getPortFromHostPortString

    public static int getPortFromHostPortString(String addr) throws IllegalArgumentException
    Get port as integer from host port string like host:port.
    Parameters:
      `addr` - host + port string like host:port.
    Returns:
      an integer value representing the port.
    Throws:
      `IllegalArgumentException` - if the input is not in the correct format.

  - ### getLocalInetAddress

    public static InetAddress getLocalInetAddress(String host) throws SocketException
    Checks if `host` is a local host name and return `InetAddress` corresponding to that address.
    Parameters:
      `host` - the specified host
    Returns:
      a valid local `InetAddress` or null
    Throws:
      `SocketException` - if an I/O error occurs

  - ### isLocalAddress

    public static boolean isLocalAddress(InetAddress addr)
    Given an InetAddress, checks to see if the address is a local address, by comparing the address with all the interfaces on the node.
    Parameters:
      `addr` - address to check if it is local node's address
    Returns:
      true if the address corresponds to the local node

  - ### wrapException

    public static IOException wrapException(String destHost, int destPort, String localHost, int localPort, IOException exception)
    Take an IOException , the local host port and remote host port details and return an IOException with the input exception as the cause and also include the host details. The new exception provides the stack trace of the place where the exception is thrown and some extra diagnostics information. If the exception is of type BindException, ConnectException, UnknownHostException, SocketTimeoutException or has a String constructor, return a new one of the same type; Otherwise return an IOException.
    Parameters:
      `destHost` - target host (nullable)
      `destPort` - target port
      `localHost` - local host (nullable)
      `localPort` - local port
      `exception` - the caught exception.
    Returns:
      an exception to throw

  - ### addNodeNameToIOException

    public static IOException addNodeNameToIOException(IOException ioe, String nodeName)
    Return an @`IOException` of the same type as the input exception but with a modified exception message that includes the node name.
    Parameters:
      `ioe` - existing exception.
      `nodeName` - name of the node.
    Returns:
      IOException

  - ### isValidSubnet

    public static boolean isValidSubnet(String subnet)
    isValidSubnet.
    Parameters:
      `subnet` - subnet.
    Returns:
      true if the given string is a subnet specified using CIDR notation, false otherwise

  - ### getIPs

    public static List<InetAddress> getIPs(String subnet, boolean returnSubinterfaces)
    Return an InetAddress for each interface that matches the given subnet specified using CIDR notation.
    Parameters:
      `subnet` - subnet specified using CIDR notation
      `returnSubinterfaces` - whether to return IPs associated with subinterfaces
    Returns:
      ips.
    Throws:
      `IllegalArgumentException` - if subnet is invalid

  - ### getFreeSocketPort

    public static int getFreeSocketPort()
    Return a free port number. There is no guarantee it will remain free, so it should be used immediately.
    Returns:
      A free port for binding a local socket

  - ### getFreeSocketPorts

    public static Set<Integer> getFreeSocketPorts(int numOfPorts)
    Return free ports. There is no guarantee they will remain free, so ports should be used immediately. The number of free ports returned by this method should match argument `numOfPorts`. Num of ports provided in the argument should not exceed 25.
    Parameters:
      `numOfPorts` - Number of free ports to acquire.
    Returns:
      Free ports for binding a local socket.

  - ### bindToLocalAddress

    public static InetAddress bindToLocalAddress(InetAddress localAddr, boolean bindWildCardAddress)
    Return an @`InetAddress` to bind to. If bindWildCardAddress is true then returns null.
    Parameters:
      `localAddr` - local addr.
      `bindWildCardAddress` - bind wildcard address.
    Returns:
      InetAddress