In radiation therapy of abdominal targets, optimal tumor irradiation can be challenging due to intrafractional motion. Current target localization methods are mainly indirect, surrogate-based and the patient is exposed to additional radiation due to X-ray imaging. In contrast, 4D ultrasound (4DUS) imaging provides volumetric images of soft tissue tumors in real-time without ionizing radiation, facilitating a non-invasive, direct tracking method. In this study, the target was defined by features located in its local neighborhood. Features were extracted using the FAST detector and the BRISK descriptor, which were extended to 3D. To account for anatomical variability, a feature library was generated that contains manually annotated target information and relative locations of the features. During tracking, features were extracted from the current 4DUS volume and compared to the feature library. Recognized features are used to estimate feature position and shape. The developed method was evaluated in 4DUS sequences of the liver of three healthy subjects. For each dataset, a target was defined and manually contoured in a training and a test sequence. Training was used for library creation, the test sequence for target tracking. The target estimations are compared to the annotations to quantify a tracking error. The results show that binary feature libraries can be used for robust target localization in 4DUS data of the liver and could potentially serve as a tracking method less sensitive to target deformation.