org.apache.nutch.protocol.ftp

## Class Ftp

- java.lang.Object
- - org.apache.nutch.protocol.ftp.Ftp

- All Implemented Interfaces:
    Configurable, Pluggable, Protocol

      public class Ftp
      extends Object
      implements Protocol

  This class is a protocol plugin used for ftp: scheme. It creates `FtpResponse` object and gets the content of the url from it. Configurable parameters are `ftp.username`, `ftp.password`, `ftp.content.limit`, `ftp.timeout`, `ftp.server.timeout`, `ftp.password`, `ftp.keep.connection` and `ftp.follow.talk` . For details see "FTP properties" section in `nutch-default.xml`.

- - ### Field Summary

    Fields
    Modifier and Type | Field and Description
    --- | ---
    `protected static org.slf4j.Logger` | `LOG`

    - ### Fields inherited from interface org.apache.nutch.protocol.Protocol

      `X_POINT_ID`

  - ### Constructor Summary

    Constructors
    Constructor and Description
    `Ftp()`

  - ### Method Summary

    All Methods Static Methods Instance Methods Concrete Methods
    Modifier and Type | Method and Description
    --- | ---
    `protected void` | `finalize()`
    `int` | `getBufferSize()`
    `Configuration` | `getConf()` Get the `Configuration` object
    `ProtocolOutput` | `getProtocolOutput(Text url, CrawlDatum datum)` Creates a `FtpResponse` object corresponding to the url and returns a `ProtocolOutput` object as per the content received
    `crawlercommons.robots.BaseRobotRules` | `getRobotRules(Text url, CrawlDatum datum, List<Content> robotsTxtContent)` Get the robots rules for a given url
    `static void` | `main(String[] args)` For debugging.
    `void` | `setConf(Configuration conf)` Set the `Configuration` object
    `void` | `setFollowTalk(boolean followTalk)` Set followTalk
    `void` | `setKeepConnection(boolean keepConnection)` Set keepConnection
    `void` | `setMaxContentLength(int length)` Set the point at which content is truncated.
    `void` | `setTimeout(int to)` Set the timeout.

    - ### Methods inherited from class java.lang.Object

      `clone, equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- - ### Field Detail

    - #### LOG

          protected static final org.slf4j.Logger LOG

  - ### Constructor Detail

    - #### Ftp

          public Ftp()

  - ### Method Detail

    - #### setTimeout

          public void setTimeout(int to)

      Set the timeout.

    - #### setMaxContentLength

          public void setMaxContentLength(int length)

      Set the point at which content is truncated.

    - #### setFollowTalk

          public void setFollowTalk(boolean followTalk)

      Set followTalk

    - #### setKeepConnection

          public void setKeepConnection(boolean keepConnection)

      Set keepConnection

    - #### getProtocolOutput

          public ProtocolOutput getProtocolOutput(Text url,
                                                  CrawlDatum datum)

      Creates a `FtpResponse` object corresponding to the url and returns a `ProtocolOutput` object as per the content received
      Specified by:
        `getProtocolOutput` in interface `Protocol`
      Parameters:
        `url` - Text containing the ftp url
        `datum` - The CrawlDatum object corresponding to the url
      Returns:
        `ProtocolOutput` object for the url

    - #### finalize

          protected void finalize()

      Overrides:
        `finalize` in class `Object`

    - #### main

          public static void main(String[] args)
                           throws Exception

      For debugging.
      Throws:
        `Exception`

    - #### setConf

          public void setConf(Configuration conf)

      Set the `Configuration` object
      Specified by:
        `setConf` in interface `Configurable`

    - #### getConf

          public Configuration getConf()

      Get the `Configuration` object
      Specified by:
        `getConf` in interface `Configurable`

    - #### getRobotRules

          public crawlercommons.robots.BaseRobotRules getRobotRules(Text url,
                                                                    CrawlDatum datum,
                                                                    List<Content> robotsTxtContent)

      Get the robots rules for a given url
      Specified by:
        `getRobotRules` in interface `Protocol`
      Parameters:
        `url` - URL to check
        `datum` - page datum
        `robotsTxtContent` - container to store responses when fetching the robots.txt file for debugging or archival purposes. Instead of a robots.txt file, it may include redirects or an error page (404, etc.). Response `Content` is appended to the passed list. If null is passed nothing is stored.
      Returns:
        robot rules (specific for this URL or default), never null

    - #### getBufferSize

          public int getBufferSize()

Copyright © 2021 The Apache Software Foundation